Confidence Intervals for the Mean Response

What a Mean-Response Interval Describes

A fitted regression value estimates the average outcome for observations with a particular predictor value or combination of predictor values. That average is not known exactly because the regression coefficients were estimated from a sample. A confidence interval quantifies this estimation uncertainty. It does not describe the range in which an individual outcome is expected to fall; individual outcomes also vary around their population mean.

For example, a hypothetical insurer might model the average claim cost for policies with specified characteristics. An interval for that average concerns the population mean claim cost for that group. It is not an interval for the claim cost of one policyholder. The distinction matters whenever the question asks whether uncertainty concerns a group mean or an individual result.

Definition: The population mean response at predictor setting \(x_0\) is the expected value of the response for observations with that setting. In simple linear regression, it is \(\mu_Y(x_0)=\beta_0+\beta_1x_0\). Its estimate is the fitted value \(\hat{Y}_0=\hat{\beta}_0+\hat{\beta}_1x_0\). A confidence interval for \(\mu_Y(x_0)\) accounts for the sampling uncertainty in the estimated regression coefficients.

Model, Assumptions, and the Interval Formula

For simple linear regression, suppose the observations satisfy \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), for \(i=1,\ldots,n\). To use the usual t-based confidence interval, assume the predictor values are fixed (or condition on their observed values), the errors are independent and normally distributed with mean zero and constant variance \(\sigma^2\), and the regression model is linear in its coefficients. The estimates \(\hat{\beta}_0\) and \(\hat{\beta}_1\) are obtained by least squares. Write \(s=\sqrt{\mathrm{MSE}}\) for the estimated error standard deviation, where \(\mathrm{MSE}\) is the residual mean square from the regression ANOVA table.

Let \(\bar{x}\) be the sample mean of the predictor, and let \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\). With unknown \(\sigma\) estimated by \(s\), the standard error of the fitted mean at \(x_0\) is

SE(\hat{Y}_0)=s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

The first term inside the square root reflects estimation of the intercept and the second reflects how far \(x_0\) is from the center of the observed predictor values. Given the assumptions above, a \(100(1-\alpha)\%\) confidence interval for the population mean response is

\hat{Y}_0\ \pm\ t_{1-\alpha/2,\ n-2}\, s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

Here \(t_{1-\alpha/2,\ n-2}\) is the corresponding quantile of the t distribution with \(n-2\) degrees of freedom. The interval is centered at the fitted mean. It is typically most precise near \(\bar{x}\), where the distance term is zero, and less precise farther from the center. This formula should not be used as though it guarantees reliable inference far outside the range of observed predictor values: the linear model may not describe such extrapolations well.

Why the Standard Error Has This Form

The standard error follows from the variance of a linear combination of the estimated regression coefficients. In simple regression, the fitted mean is \(\hat{Y}_0=\hat{\beta}_0+x_0\hat{\beta}_1\). Under the stated model, the coefficient estimates have variances \(\operatorname{Var}(\hat{\beta}_0)=\sigma^2(1/n+\bar{x}^2/S_{xx})\) and \(\operatorname{Var}(\hat{\beta}_1)=\sigma^2/S_{xx}\), with covariance \(\operatorname{Cov}(\hat{\beta}_0,\hat{\beta}_1)=-\sigma^2\bar{x}/S_{xx}\). Applying the variance formula for a sum gives

\begin{aligned} \operatorname{Var}(\hat{Y}_0) &=\operatorname{Var}(\hat{\beta}_0)+x_0^2\operatorname{Var}(\hat{\beta}_1) +2x_0\operatorname{Cov}(\hat{\beta}_0,\hat{\beta}_1)\\ &=\sigma^2\left(\frac{1}{n}+\frac{\bar{x}^2}{S_{xx}} +\frac{x_0^2}{S_{xx}}-\frac{2x_0\bar{x}}{S_{xx}}\right)\\ &=\sigma^2\left(\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right). \end{aligned}

Replacing \(\sigma\) with \(s\) gives the standard error formula. Under normal errors, the fitted-mean error divided by its estimated standard error has a t distribution with \(n-2\) degrees of freedom. This justifies using the t critical value rather than treating the standard error as known.

Worked Example: A Mean Response at the Predictor Center

A hypothetical study relates a service score \(Y\) to a numerical predictor \(x\). The regression summary gives \(n=8\), \(\bar{x}=4\), \(S_{xx}=20\), \(s=2\), and fitted value \(\hat{Y}_0=12.4\) at \(x_0=4\). Construct a 95% confidence interval for the population mean response at \(x_0=4\). Use \(t_{0.975,6}=2.447\).

Method. Use the simple-regression mean-response interval. Since \(x_0=\bar{x}\), the distance term is zero:

\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}} =\frac{1}{8}+\frac{(4-4)^2}{20}=0.125.

The standard error is \(2\sqrt{0.125}=2(0.353553)=0.707107\). The margin of error is \(2.447(0.707107)=1.730290\). Therefore, the interval is \(12.4\pm1.730290\), or \((10.669710,\ 14.130290)\). To three decimal places, it is \((10.670,\ 14.130)\).

Check. The interval is centered at 12.4: its unrounded endpoints are each 1.730290 from the fitted mean. The predictor is at its sample center, so the distance contribution to uncertainty is zero. A common mistake is to treat this as zero uncertainty; the \(1/n\) term remains because the mean response is estimated from a finite sample.

Mean-Response Uncertainty Versus Individual-Outcome Uncertainty

At a fixed predictor setting, an individual response has both the uncertainty in the estimated mean and its own random error around that mean. Consequently, its prediction standard error is larger. For simple regression, the estimated standard error for predicting one new individual response at \(x_0\) is

s\sqrt{1+\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

The additional \(1\) represents the new observation’s residual variation. The confidence interval for the population mean uses only the leverage expression \(1/n+(x_0-\bar{x})^2/S_{xx}\); it must not include that extra \(1\). Conversely, an individual prediction interval must include it. Both intervals are centered at the same fitted value, but an individual prediction interval is wider under the same model and confidence level.

Worked Example: Comparing the Mean and an Individual Outcome

A hypothetical model for repair duration has \(n=10\), \(\bar{x}=5\), \(S_{xx}=30\), \(s=3\), and fitted value \(\hat{Y}_0=18.6\) at \(x_0=8\). Construct the 95% confidence interval for the mean response, then compare it with the 95% prediction interval for one new outcome. Use \(t_{0.975,8}=2.306\).

Method. First calculate the leverage expression shared by the two standard errors:

h_0=\frac{1}{10}+\frac{(8-5)^2}{30} =0.1+\frac{9}{30}=0.4.

For the mean response, \(SE=3\sqrt{0.4}=1.897367\). Its margin of error is \(2.306(1.897367)=4.375327\), so the 95% confidence interval is \(18.6\pm4.375327=(14.224673,\ 22.975327)\). To three decimal places, this is \((14.225,\ 22.975)\).

For one new outcome, include the additional \(1\): \(SE_{\mathrm{pred}}=3\sqrt{1+0.4}=3\sqrt{1.4}=3.549648\). The prediction margin is \(2.306(3.549648)=8.185488\), giving \(18.6\pm8.185488=(10.414512,\ 26.785488)\), or \((10.415,\ 26.785)\) to three decimal places.

Check. Both intervals are centered at 18.6, and the prediction interval is wider. The common error this comparison avoids is adding the \(1\) under the square root when estimating uncertainty in the mean, or omitting it when predicting an individual response.

Extension to Multiple Regression

For a multiple regression with \(p\) coefficients, including the intercept, let \(\mathbf{x}_0\) be the vector of predictor values at the setting of interest, including a leading 1 for the intercept. Let \(\mathbf{X}\) be the design matrix, with one row per observation, and suppose it has full column rank. Under the same independent, normal, constant-variance error assumptions, the fitted mean is \(\hat{Y}_0=\mathbf{x}_0^\mathsf{T}\hat{\boldsymbol{\beta}}\), and its estimated standard error is \(s\sqrt{\mathbf{x}_0^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{x}_0}\). The t interval uses \(n-p\) degrees of freedom. The quadratic expression depends on the entire predictor setting and the design, not just on one predictor’s distance from its sample mean.

Worked Example: A Multiple-Regression Mean Response

A hypothetical model has \(n=15\) observations and three coefficients. At a specified predictor setting, \(\mathbf{x}_0=(1,2,1)^\mathsf{T}\), the estimated coefficient vector is \(\hat{\boldsymbol{\beta}}=(4,1.2,-0.5)^\mathsf{T}\), \(s=2\), and the relevant matrix is

(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1} = \begin{pmatrix} 0.04&0&0\\ 0&0.01&0\\ 0&0&0.02 \end{pmatrix}.

Construct a 95% confidence interval for the mean response at this setting, using \(t_{0.975,12}=2.179\).

Method. Calculate the fitted mean by taking the coefficient combination: \(4(1)+1.2(2)-0.5(1)=4+2.4-0.5=5.9\). The quadratic expression is

\mathbf{x}_0^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{x}_0 =1^2(0.04)+2^2(0.01)+1^2(0.02)=0.04+0.04+0.02=0.10.

Thus \(SE=2\sqrt{0.10}=0.632456\). There are \(15-3=12\) residual degrees of freedom. The margin of error is \(2.179(0.632456)=1.378121\), so the interval is \(5.9\pm1.378121=(4.521879,\ 7.278121)\). To three decimal places, it is \((4.522,\ 7.278)\).

Check. The interval is centered at the calculated fitted mean, 5.9, and uses \(n-p=12\), not \(n-2\), degrees of freedom. A common mistake is to apply the simple-regression degrees of freedom or to use a single predictor’s leverage formula in a multiple-regression setting.

Interpretation and Common Traps

A 95% confidence procedure has 95% coverage in repeated samples when the model assumptions hold: across repeated applications, about 95% of the intervals constructed by this method contain the fixed population mean response at the specified predictor setting. After an interval has been calculated, it is not correct to say that the fixed mean has a 95% probability of lying in that particular interval. The interval either contains the fixed mean or it does not; the confidence level describes the procedure.

Before calculating, identify whether the target is a population mean or one individual outcome, specify the predictor setting, and confirm which regression model and residual degrees of freedom apply. Then compute the fitted value, the appropriate standard error, and the t-based margin of error. Keep the model’s assumptions in view: a narrow interval does not compensate for a misspecified relationship, dependent errors, or nonconstant error variance.

Adding the prediction term to a mean interval. Do not add \(1\) inside the square root for a confidence interval for the mean response.
Ignoring predictor location. In simple regression, the standard error generally increases as \(x_0\) moves farther from \(\bar{x}\).
Using the wrong degrees of freedom. Simple regression has \(n-2\) residual degrees of freedom; multiple regression with \(p\) coefficients has \(n-p\).
Interpreting confidence as individual coverage. A mean-response interval is not designed to contain a particular future outcome.

Key takeaway: A confidence interval for the mean response is centered at the fitted regression value and uses the uncertainty in the estimated mean. An individual prediction interval includes additional residual variation and is therefore wider under the same model and confidence level.

Check Your Understanding

For each problem, distinguish the target from the fitted value and use the stated model information.

A simple regression has \(n=16\), \(\bar{x}=10\), \(S_{xx}=80\), \(s=4\), and fitted mean 30 at \(x_0=10\). Find a 95% confidence interval for the mean response using \(t_{0.975,14}=2.145\). (Answer: \((27.855,\ 32.145)\).)
A simple regression has \(n=25\), \(\bar{x}=4\), \(S_{xx}=100\), \(s=5\), and fitted mean 14 at \(x_0=9\). Find a 95% confidence interval for the mean response using \(t_{0.975,23}=2.069\), reporting endpoints to three decimals. (Answer: \((8.429,\ 19.571)\).)
At a predictor setting, a regression has fitted mean 6, leverage expression \(1/n+(x_0-\bar{x})^2/S_{xx}=0.25\), \(s=2\), and 18 residual degrees of freedom. Using \(t_{0.975,18}=2.101\), find both the 95% mean-response interval and the 95% individual prediction interval, to three decimals. (Answer: Mean response \((3.899,\ 8.101)\); individual prediction \((1.302,\ 10.698)\).)
A multiple regression has \(n=20\), \(p=4\) coefficients, fitted mean 12 at a specified setting, \(s=3\), and \(\mathbf{x}_0^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{x}_0=0.09\). Find the 95% mean-response interval using \(t_{0.975,16}=2.120\). (Answer: \((10.092,\ 13.908)\).)
In simple regression, explain why the mean-response standard error is smallest at \(x_0=\bar{x}\), relative to other predictor settings within the model. (Answer: The term \((x_0-\bar{x})^2/S_{xx}\) is nonnegative and equals zero at \(x_0=\bar{x}\), so the leverage expression, and hence its square-root standard error, is minimized there.)

What You'll Learn