Deriving a Confidence Interval for the Mean Response

Purpose and Setup

In simple linear regression, the fitted line estimates the average response associated with each predictor value. At a specified value \(x_0\), the fitted value is an estimate of the population mean response there. A confidence interval adds a measure of sampling uncertainty to that estimate. This tutorial derives the exact pointwise t interval under the normal-error regression model and shows how to calculate it.

The interval concerns the mean response at one fixed predictor value, not the response of a particular future individual. The distinction matters because the uncertainty in a mean is different from the uncertainty in an individual outcome.

Model and assumptions: For fixed observed predictor values \(x_1,\ldots,x_n\), suppose \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), where the errors \(\varepsilon_i\) are independent and each has a normal distribution with mean 0 and variance \(\sigma^2\). The variance is the same for every observation, and the predictor values are not all identical. We require \(n\geq 3\) so that the residual variance can be estimated with positive degrees of freedom.

The population mean response at \(x_0\) is \(\mu(x_0)=\beta_0+\beta_1x_0\). The least-squares estimates are \(b_0\) and \(b_1\), giving fitted mean \(\hat{Y}(x_0)=b_0+b_1x_0\). Define \(\bar{x}\) as the sample mean of the predictor values and \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\). The residual sum of squares is \(SSE=\sum_{i=1}^n(Y_i-b_0-b_1x_i)^2\), and the estimated error variance and residual standard deviation are

s^2=\frac{SSE}{n-2},\qquad s=\sqrt{s^2}.

The \(n-2\) degrees of freedom arise because the fitted line estimates two parameters, the intercept and the slope.

Why the Standard Error Has This Form

The standard error of the fitted mean depends on both the amount of residual variation and the location of \(x_0\) relative to the observed predictors. The result is

SE\{\hat{Y}(x_0)\} =s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

To see why, write the fitted value in terms of the sample mean response \(\bar{Y}\) and the estimated slope:

\hat{Y}(x_0)=\bar{Y}+b_1(x_0-\bar{x}).

Under the stated model, \(\operatorname{Var}(\bar{Y})=\sigma^2/n\) and \(\operatorname{Var}(b_1)=\sigma^2/S_{xx}\). The covariance between \(\bar{Y}\) and \(b_1\) is zero: the slope is a weighted sum of the responses with weights proportional to \(x_i-\bar{x}\), whose sum is zero. Therefore, the variance of the fitted mean is

\begin{aligned} \operatorname{Var}\{\hat{Y}(x_0)\} &=\operatorname{Var}(\bar{Y})+(x_0-\bar{x})^2\operatorname{Var}(b_1)\\ &=\sigma^2\left(\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right). \end{aligned}

Replacing the unknown \(\sigma\) by \(s\) gives the estimated standard error above. The first term reflects uncertainty in the overall response level. The second reflects uncertainty in the estimated slope, which matters more as \(x_0\) moves farther from \(\bar{x}\).

Deriving the Pointwise t Interval

Under the normal-error assumptions, the fitted mean is normally distributed around \(\mu(x_0)\). Also, \(SSE/\sigma^2\) has a chi-square distribution with \(n-2\) degrees of freedom and is independent of the fitted mean. Consequently, replacing \(\sigma\) with \(s\) in the standardized error produces a t statistic with \(n-2\) degrees of freedom:

\frac{\hat{Y}(x_0)-\mu(x_0)} {s\sqrt{1/n+(x_0-\bar{x})^2/S_{xx}}} \sim t_{n-2}.

Let \(t_{1-\alpha/2,n-2}\) be the value with probability \(1-\alpha/2\) to its left in a t distribution with \(n-2\) degrees of freedom. The central \(1-\alpha\) probability of this t distribution lies between \(-t_{1-\alpha/2,n-2}\) and \(t_{1-\alpha/2,n-2}\). Rearranging that probability statement for \(\mu(x_0)\) gives the confidence interval:

\hat{Y}(x_0)\ \pm\ t_{1-\alpha/2,n-2}\, s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}.

This is a pointwise interval: its stated coverage applies to the mean at the specified \(x_0\). It is exact when the model assumptions hold. Without normal errors, the t procedure is not generally exact, although it may be used as an approximation in some settings.

Identify the model quantities.
Record \(n\), \(\bar{x}\), \(S_{xx}\), \(SSE\), and the requested predictor value \(x_0\), along with the fitted line or fitted value.

Estimate residual variation.
Calculate \(s=\sqrt{SSE/(n-2)}\), using residual degrees of freedom rather than \(n-1\).

Calculate the standard error and critical value.
Use the factor involving \(x_0-\bar{x}\), and use the t critical value with \(n-2\) degrees of freedom.

Form and check the limits.
Add and subtract the margin of error from \(\hat{Y}(x_0)\). The interval should be centered on that fitted mean.

Worked Example: Constructing an Interval from Raw Data

A hypothetical study records a response \(Y\) for five fixed predictor settings. The observed pairs are \((0,2),(1,3),(2,5),(3,4),(4,6)\). Assuming the simple linear regression model and its normal, independent, constant-variance errors, find a 95% confidence interval for the mean response at \(x_0=2.5\). Use three decimal places for the final limits.

Method and summaries. First calculate the means: \(\bar{x}=(0+1+2+3+4)/5=2\), and \(\bar{Y}=(2+3+5+4+6)/5=4\). The predictor deviations are \(-2,-1,0,1,2\), so

S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10.

The response deviations from 4 are \(-2,-1,1,0,2\). Their products with the corresponding predictor deviations sum to \((-2)(-2)+(-1)(-1)+(0)(1)+(1)(0)+(2)(2)=9\). Thus \(b_1=9/10=0.9\), and \(b_0=\bar{Y}-b_1\bar{x}=4-(0.9)(2)=2.2\).

The fitted response at \(x_0=2.5\) is \(\hat{Y}(2.5)=2.2+(0.9)(2.5)=4.45\). The fitted values at the observed predictor values \(0,1,2,3,4\) are \(2.2,3.1,4.0,4.9,5.8\). The corresponding residuals are \(-0.2,-0.1,1.0,-0.9,0.2\), so

$$ SSE=(-0.2)^2+(-0.1)^2+(1.0)^2+(-0.9)^2+(0.2)^2 =0.04+0.01+1.00+0.81+0.04=1.90. $$

There are \(n-2=5-2=3\) residual degrees of freedom. Therefore, \(s^2=1.90/3=0.633333\), and \(s=\sqrt{0.633333}=0.795822\). For a two-sided 95% interval, \(t_{0.975,3}=3.1824\). The standard-error factor is \(\sqrt{1/5+(2.5-2)^2/10}=\sqrt{0.2+0.025}=\sqrt{0.225}=0.474342\). Hence the estimated standard error is \(0.795822(0.474342)=0.377492\), and the margin of error is \(3.1824(0.377492)=1.201347\).

The limits are \(4.45-1.201347=3.248653\) and \(4.45+1.201347=5.651347\). Rounded to three decimal places, the interval is \((3.249,5.651)\). It is centered on 4.45, as required. Common mistake avoided: using a t critical value with \(n-1=4\) degrees of freedom would use the wrong residual degrees of freedom; the fitted intercept and slope leave \(n-2=3\).

Worked Example: Using Regression Summary Statistics

A hypothetical analysis reports \(n=8\), \(\bar{x}=10\), \(\bar{Y}=25\), \(S_{xx}=56\), slope estimate \(b_1=1.5\), and \(SSE=24\). Find a 90% pointwise confidence interval for the mean response at \(x_0=12\). Round the final limits to three decimal places.

Method. Recover the intercept from \(b_0=\bar{Y}-b_1\bar{x}=25-(1.5)(10)=10\). The fitted mean at 12 is \(10+(1.5)(12)=28\). The residual variance estimate is \(s^2=24/(8-2)=24/6=4\), so \(s=2\).

For a 90% two-sided interval and 6 degrees of freedom, \(t_{0.95,6}=1.9432\). Since \(x_0-\bar{x}=12-10=2\), the standard-error factor is \(\sqrt{1/8+2^2/56}=\sqrt{0.125+0.0714286}=\sqrt{0.1964286}=0.443203\). Thus \(SE=2(0.443203)=0.886405\), and the margin of error is \(1.9432(0.886405)=1.72245\).

The confidence limits are \(28-1.72245=26.27755\) and \(28+1.72245=29.72245\). Rounded to three decimal places, the interval is \((26.278,29.722)\). Common mistake avoided: the confidence level is 90%, so the critical value uses 0.95 as its cumulative probability, not 0.90.

Worked Example: Evaluating the Mean at the Predictor Average

For a hypothetical regression, suppose \(n=10\), \(\bar{x}=4\), \(S_{xx}=30\), \(b_0=7\), \(b_1=-0.4\), and \(SSE=32\). Find a 95% pointwise confidence interval for the mean response at \(x_0=4\). Round the final limits to three decimal places.

Method. The fitted mean is \(7+(-0.4)(4)=5.4\). The residual degrees of freedom are \(10-2=8\), so \(s^2=32/8=4\) and \(s=2\). The 95% critical value is \(t_{0.975,8}=2.3060\).

Because \(x_0=\bar{x}\), the squared-distance term is zero. The standard error is therefore \(2\sqrt{1/10+0^2/30}=2\sqrt{0.1}=0.632456\). The margin of error is \(2.3060(0.632456)=1.45844\). The limits are \(5.4-1.45844=3.94156\) and \(5.4+1.45844=6.85844\). Rounded to three decimal places, the interval is \((3.942,6.858)\). Common mistake avoided: do not add a distance term when \(x_0=\bar{x}\); at the predictor average, that term is exactly zero.

Common Calculation Traps

Using the wrong degrees of freedom. The residual standard deviation uses \(SSE/(n-2)\), and the t critical value also has \(n-2\) degrees of freedom. The two fitted regression coefficients account for the difference from \(n\).
Leaving out the location term. The standard error is not always \(s/\sqrt{n}\). That is its value only when \(x_0=\bar{x}\); otherwise include \((x_0-\bar{x})^2/S_{xx}\).
Using a z critical value automatically. When \(\sigma\) is unknown and estimated by \(s\), the exact interval under the normal-error model uses a t critical value.
Changing the target. This formula estimates the mean response. A future individual response has additional uncertainty from its own error term; it requires a different interval.
Ignoring the assumptions. The exact t result relies on independent normal errors with a common variance and fixed predictor values. If those assumptions do not hold, the stated exact coverage need not hold.

Key result: At a specified \(x_0\), the pointwise \(100(1-\alpha)\%\) confidence interval for the regression mean is \(\hat{Y}(x_0)\pm t_{1-\alpha/2,n-2}s\sqrt{1/n+(x_0-\bar{x})^2/S_{xx}}\). The fitted mean is the center; the predictor's distance from \(\bar{x}\) affects its standard error.

Check Your Understanding

For each problem, use the simple linear regression model with independent normal errors of constant variance. Unless stated otherwise, report final interval limits to three decimal places.

A regression has \(n=12\), \(\bar{x}=5\), \(S_{xx}=44\), \(s=3\), and fitted mean \(\hat{Y}(7)=18\). Find the 95% pointwise confidence interval. Use \(t_{0.975,10}=2.2281\). (Answer: \((15.210,20.790)\))
A regression summary gives \(n=9\), \(\bar{x}=20\), \(S_{xx}=72\), \(\bar{Y}=40\), \(SSE=28\), and \(b_1=0.8\). Find the 90% interval at \(x_0=20\). Use \(t_{0.95,7}=1.8946\). (Answer: \((38.737,41.263)\))
For \(n=6\), \(\bar{x}=2\), \(S_{xx}=10\), fitted mean \(\hat{Y}(3)=11\), and \(SSE=12\), calculate a 95% interval at \(x_0=3\). Use \(t_{0.975,4}=2.7764\). (Answer: \((8.517,13.483)\))
A fitted mean at \(\bar{x}=6\) is 12 for a regression with \(n=16\), \(SSE=56\), and \(S_{xx}=80\). Find a 95% interval at \(x_0=6\). Use \(t_{0.975,14}=2.1448\). (Answer: \((10.928,13.072)\))
A regression has \(n=7\), \(\bar{x}=3\), \(S_{xx}=14\), fitted mean \(\hat{Y}(5)=9\), and \(SSE=20\). Find a 90% interval at \(x_0=5\). Use \(t_{0.95,5}=2.0150\). (Answer: \((6.362,11.638)\))

What You'll Learn