Uncertainty in the Estimated Mean Response

Why the Predictor Value Matters

A fitted regression line gives an estimated mean response at every predictor value. But those estimates are not equally precise. The line is usually estimated most accurately near the center of the observed predictor values; uncertainty grows as the value of interest moves away from that center. This tutorial shows how to calculate that uncertainty and form a confidence interval for the mean response at a specified predictor value.

The key inputs are the sample size, the spread of the observed predictor values, the estimated error standard deviation, and the distance between the chosen predictor value and the predictor mean. A confidence interval for the mean response describes uncertainty in the location of the regression line, not the variation of a new individual response around that line.

Definition: In the simple linear regression model $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$, the conditional mean response at predictor value $x_0$ is $E[Y\mid x_0]=\beta_0+\beta_1x_0$. Its estimate is the fitted value $\hat{Y}(x_0)=\hat{\beta}_0+\hat{\beta}_1x_0$. A confidence interval for the mean response gives a range of plausible values for $\beta_0+\beta_1x_0$, based on the observed sample.

Standard Error of the Estimated Mean Response

Suppose the observed predictor values are treated as fixed, and the errors are independent with mean zero and common variance $\sigma^2$. Write $\bar{x}$ for the sample mean of the predictors and $S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2$ for their sum of squared deviations. The residual standard error is $s=\sqrt{\mathrm{SSE}/(n-2)}$, where $\mathrm{SSE}=\sum_{i=1}^n(Y_i-\hat{Y}_i)^2$. The $n-2$ degrees of freedom account for estimating the two regression coefficients.

Under these conditions, the standard error of the estimated mean response at $x_0$ is

\operatorname{SE}\{\hat{Y}(x_0)\} =s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

The first term inside the square root, $1/n$, represents uncertainty associated with estimating the response level at the center of the observed predictor values. The second term accounts for the additional uncertainty from estimating the slope when the target value is away from that center. If $x_0=\bar{x}$, the second term is zero, and the standard error is $s/\sqrt{n}$.

Why the Formula Has a Distance Term

Centering the predictor makes the reason for the distance term visible. The fitted response can be written as $\hat{Y}(x_0)=\bar{Y}+\hat{\beta}_1(x_0-\bar{x})$. In the model with fixed predictors and independent errors of variance $\sigma^2$, the sample mean response has variance $\sigma^2/n$, and the least-squares slope has variance $\sigma^2/S_{xx}$.

The covariance between $\bar{Y}$ and $\hat{\beta}_1$ is zero: the contribution of each error to the covariance is proportional to $x_i-\bar{x}$, and those deviations sum to zero. Thus, the variance of the fitted mean is the sum of the two component variances:

\begin{aligned} \operatorname{Var}\{\hat{Y}(x_0)\} &=\operatorname{Var}(\bar{Y}) +(x_0-\bar{x})^2\operatorname{Var}(\hat{\beta}_1)\\ &=\frac{\sigma^2}{n} +\frac{\sigma^2(x_0-\bar{x})^2}{S_{xx}}. \end{aligned}

Taking the square root gives the standard deviation of the estimated mean response. Since $\sigma$ is not known, it is replaced by $s$, giving the estimated standard error above. This derivation also explains why the distance is squared: moving the target predictor value either above or below $\bar{x}$ increases variance by the same amount for equal distances.

Confidence Interval for the Mean Response

If, in addition, the regression errors are normally distributed, then the standardized estimation error, with $\sigma$ replaced by $s$, follows a $t$ distribution with $n-2$ degrees of freedom. The normality condition makes this a finite-sample exact interval; without normal errors, the same form may be used as an approximation in suitable large samples, but it is not generally an exact small-sample result.

Theorem: Under a linear conditional mean, independent errors with mean zero and common variance, and normally distributed errors, a two-sided $100(1-\alpha)\%$ confidence interval for the mean response at $x_0$ is $$ \hat{Y}(x_0)\ \pm\ t_{1-\alpha/2,\ n-2}\, s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}. $$ Here $t_{1-\alpha/2,\ n-2}$ is the $1-\alpha/2$ quantile of the $t$ distribution with $n-2$ degrees of freedom.

The interval is centered on the fitted mean response. To calculate it, find the fitted value, calculate the standard error using the target’s distance from $\bar{x}$, obtain the $t$ critical value with $n-2$ degrees of freedom, and multiply the standard error by that critical value. The resulting margin of error is added to and subtracted from the fitted value.

Worked Example: A Target Near the Predictor Mean

A hypothetical study of $n=8$ insurance service offices models average weekly processing time $Y$, in hours, against a workload index $x$. The predictor summaries are $\bar{x}=4$ and $S_{xx}=28$. The residual standard error is $s=2$ hours. The fitted mean processing time at $x_0=4.5$ is $\hat{Y}(4.5)=13.2$ hours. Assuming independent normal errors with common variance and a linear conditional mean, calculate a 95% confidence interval. Use $t_{0.975,6}=2.447$.

Method. Use the t interval for the mean response, with $n-2=8-2=6$ degrees of freedom. The predictor value is only $0.5$ above its sample mean.

First calculate the factor inside the square root: $\frac{1}{8}+\frac{(4.5-4)^2}{28}=\frac{1}{8}+\frac{0.25}{28}=\frac{1}{8}+\frac{1}{112}=\frac{15}{112}$.

The estimated standard error is $2\sqrt{15/112}=\sqrt{15/28}\approx0.7319$ hours. The margin of error is $2.447(0.7319)\approx1.7910$ hours.

The interval endpoints are $13.2-1.7910=11.4090$ and $13.2+1.7910=14.9910$. Rounded to two decimals, the 95% confidence interval is $(11.41,\ 14.99)$ hours.

Check and common mistake avoided. The interval is centered on 13.2, and each endpoint is about 1.79 hours from the center. Using only $s/\sqrt{n}$ would omit the small, nonzero distance term; the formula includes it even when the target is close to $\bar{x}$.

Distance, Interpolation, and Extrapolation

For fixed $n$, $s$, and $S_{xx}$, the standard error increases with $|x_0-\bar{x}|$. A target near the predictor mean therefore tends to have a narrower confidence interval than a target farther away. This describes a property of the fitted line’s estimation variance; it does not guarantee that every target inside the observed predictor range has low uncertainty. A point near an edge can still be far from $\bar{x}$.

Interpolation means estimating at a predictor value within the range of observed predictors. Extrapolation means estimating outside that range. The formula can be evaluated for either, but extrapolation deserves particular caution: beyond the data, the assumed linear relationship may not continue to hold. The calculated interval reflects uncertainty under the model; it does not automatically allow for a wrong model form or changing conditions.

Worked Example: A Target Well Away from the Predictor Mean

A hypothetical analysis of $n=10$ delivery depots relates a response $Y$, in thousands of dollars of weekly operating cost, to route complexity $x$. The summaries are $\bar{x}=12$, $S_{xx}=90$, and $s=4$ (in thousands of dollars). At $x_0=18$, the fitted mean cost is $\hat{Y}(18)=31$. Assume independent normal errors with common variance and a linear conditional mean. Find the 95% confidence interval, using $t_{0.975,8}=2.306$.

Method. Apply the mean-response interval with $n-2=8$ degrees of freedom. The target is $18-12=6$ units above the predictor mean.

The variance factor is $\frac{1}{10}+\frac{(18-12)^2}{90}=0.1+\frac{36}{90}=0.1+0.4=0.5$. Therefore the standard error is $4\sqrt{0.5}=2\sqrt{2}\approx2.8284$ thousand dollars.

The margin of error is $2.306(2.8284)\approx6.5224$ thousand dollars. The endpoints are $31-6.5224=24.4776$ and $31+6.5224=37.5224$. Rounded to two decimals, the 95% confidence interval is $(24.48,\ 37.52)$ thousand dollars.

Check and common mistake avoided. The interval is symmetric around 31, and its width reflects the target’s substantial distance from $\bar{x}$. Using $4/\sqrt{10}\approx1.2649$ as the standard error would wrongly treat this distant target as if it were at the predictor mean and would produce an unjustifiably narrow interval.

Worked Example: Comparing Precision at the Center and Away from It

A hypothetical study of $n=12$ workshops relates a response $Y$ to a machine-setting index $x$. It has $\bar{x}=20$, $S_{xx}=132$, and $s=3$. The fitted mean response is 8.4 units at $x_0=17$. Assume independent normal errors with constant variance and a linear conditional mean. Calculate the 95% confidence interval at $x_0=17$, using $t_{0.975,10}=2.228$, and compare the standard error with the one at $x_0=20$.

Method. The degrees of freedom are $12-2=10$. At $x_0=17$, the squared distance is $(17-20)^2=9$. The factor is $\frac{1}{12}+\frac{9}{132}=\frac{11}{132}+\frac{9}{132}=\frac{20}{132}=\frac{5}{33}$.

Thus, the standard error at 17 is $3\sqrt{5/33}\approx1.1677$ units. The margin of error is $2.228(1.1677)\approx2.6017$. The endpoints are $8.4-2.6017=5.7983$ and $8.4+2.6017=11.0017$. Rounded to two decimals, the 95% confidence interval is $(5.80,\ 11.00)$ units.

At the predictor mean $x_0=20$, the distance term is zero, so the standard error is $3/\sqrt{12}\approx0.8660$ units. It is smaller than the standard error at 17, as expected. The interval at 17 is centered on its own fitted value, 8.4; the comparison does not imply that the two fitted means are equal.

Check and common mistake avoided. The predictor distance contributes $9/132$ to the factor at 17, but contributes zero at 20. Do not apply the center standard error $s/\sqrt{n}$ to every target value.

What the Interval Does—and Does Not—Describe

The response in the interval is the conditional mean $\beta_0+\beta_1x_0$, not a single future observation. A new response at $x_0$ varies around that mean because of its own random error; the mean-response interval does not include that additional variation. Keep this distinction in mind when interpreting the result, without confusing uncertainty about the line with variation around the line.

The formula is conditional on the regression assumptions. If the mean relationship is not linear, if the errors have a changing variance, or if observations are dependent, the stated standard error and confidence level may not be reliable. In particular, a small numerical standard error does not protect an extrapolated estimate from a misspecified relationship. The interval quantifies uncertainty under the model, not uncertainty about whether the model itself is appropriate.

Key takeaway: For a mean-response interval at $x_0$, use the fitted value $\hat{Y}(x_0)$, the standard error $s\sqrt{1/n+(x_0-\bar{x})^2/S_{xx}}$, and a $t$ critical value with $n-2$ degrees of freedom. The squared distance term is essential: uncertainty is generally lowest at $\bar{x}$ and increases as the target moves away.

Check Your Understanding

For calculation questions, round interval endpoints to two decimals and standard errors to four decimals.

A study has $n=16$, $\bar{x}=8$, $S_{xx}=120$, $s=5$, and $\hat{Y}(8)=20$. Under the normal-error model, find the 95% mean-response interval at $x_0=8$, using $t_{0.975,14}=2.145$. (Answer: $(17.32,\ 22.68)$)
A study has $n=9$, $\bar{x}=30$, $S_{xx}=48$, $s=2.5$, and $\hat{Y}(32)=41$. Find the 95% mean-response interval at $x_0=32$, using $t_{0.975,7}=2.365$. (Answer: standard error $1.1024$; interval $(38.39,\ 43.61)$)
A fitted mean at $x_0=10$ is 22, with $n=9$, $\bar{x}=6$, $S_{xx}=24$, and $s=3$. Find the 95% interval using $t_{0.975,7}=2.365$. (Answer: standard error $\sqrt{7}\approx2.6458$; interval $(15.74,\ 28.26)$)
A regression has $n=20$, $S_{xx}=190$, and $s=3$. Compare the standard errors at $x_0=\bar{x}$ and at $x_0=\bar{x}+5$. (Answer: at $\bar{x}$, $0.6708$; at $\bar{x}+5$, $1.2784$)
A model is used at a predictor value outside the observed range. The computed standard error is larger than it would be at $\bar{x}$. Give two reasons why the resulting interval should still be interpreted cautiously. (Answer: its larger standard error reflects greater estimation uncertainty from distance to $\bar{x}$; extrapolation may violate the assumed linear relationship, a risk the interval does not account for.)

What You'll Learn