Standard Error of an Estimated Mean Response

Why the Standard Error of a Fitted Mean Matters

A fitted regression line gives an estimated mean response at a predictor value, but that estimate varies from sample to sample. The standard error of the estimated mean response measures the estimated sampling variability of that fitted mean. It is not the residual standard deviation, and it is not the standard error for predicting one future observation.

In simple linear regression, the standard error depends on three features: the estimated residual variance, the sample size, and how far the specified predictor value lies from the sample predictor mean relative to the spread of the observed predictors. These pieces explain why a fitted mean is typically estimated most precisely near the center of the observed predictor values.

Model and assumptions: For fixed predictor values \(x_1,\ldots,x_n\), suppose \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), where the errors are independent, have mean zero, and have common variance \(\sigma^2\). For exact normal-theory inference, also assume \(\varepsilon_i\sim N(0,\sigma^2)\). The standard error formula estimates the variability under the independence and constant-variance assumptions; normality is not needed for the variance calculation itself.

The Formula and Its Ingredients

Write \(\bar{x}\) for the mean of the observed predictor values and \(S_{xx}\) for their corrected sum of squares. The residual sum of squares is \(SSE=\sum_{i=1}^n(y_i-\hat{y}_i)^2\). In simple linear regression, two parameters—the intercept and slope—are estimated, so the residual mean square uses \(n-2\) degrees of freedom.

Definition: The estimated standard error of the fitted mean response at \(x_0\) is

SE\{\hat{y}(x_0)\} =s\sqrt{\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}}, \qquad s=\sqrt{MSE}=\sqrt{\frac{SSE}{n-2}}, \qquad S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2.

Here \(\hat{y}(x_0)=\hat{\beta}_0+\hat{\beta}_1x_0\) is the fitted mean response, and \(s\) is the residual standard deviation. The reported standard error is an estimate of the standard deviation of \(\hat{y}(x_0)\) across repeated samples under the stated model.

The term \(1/n\) remains even when \(x_0=\bar{x}\). The other term accounts for the additional uncertainty associated with estimating the slope when the target predictor value is away from the sample mean. The distance is squared, so equal distances on either side of \(\bar{x}\) give the same standard error.

Why the Formula Holds

The form of the formula follows from expressing the fitted value using the sample mean response and the estimated slope. Let \(\bar{Y}\) be the sample mean of the responses and \(\hat{\beta}_1\) the least-squares slope. Then \(\hat{y}(x_0)=\bar{Y}+\hat{\beta}_1(x_0-\bar{x})\). Under the fixed-predictor model, the variance of \(\bar{Y}\) is \(\sigma^2/n\), and the variance of \(\hat{\beta}_1\) is \(\sigma^2/S_{xx}\).

These two estimators have covariance zero: the slope is formed from errors multiplied by centered predictor values, whose sum is zero. Therefore, the variance of their indicated sum is the sum of the two variance contributions.

\begin{aligned} \operatorname{Var}\{\hat{y}(x_0)\} &=\operatorname{Var}\{\bar{Y}+\hat{\beta}_1(x_0-\bar{x})\}\\ &=\frac{\sigma^2}{n} +(x_0-\bar{x})^2\frac{\sigma^2}{S_{xx}}\\ &=\sigma^2\left(\frac{1}{n} +\frac{(x_0-\bar{x})^2}{S_{xx}}\right). \end{aligned}

For completeness, the zero covariance follows directly from the error model. The random parts of \(\bar{Y}\) and \(\hat{\beta}_1\) are \(\bar{\varepsilon}\) and \(\sum_i(x_i-\bar{x})\varepsilon_i/S_{xx}\). Independence gives \(\operatorname{Cov}(\bar{\varepsilon},\hat{\beta}_1)=\sigma^2\sum_i(x_i-\bar{x})/(nS_{xx})=0\), because \(\sum_i(x_i-\bar{x})=0\). Replacing the unknown \(\sigma\) by its estimate \(s\) gives the standard error formula. Under the assumptions, \(SSE/(n-2)\) estimates \(\sigma^2\) without bias.

Find the residual standard deviation.
Use \(s=\sqrt{SSE/(n-2)}\), or use a supplied value of \(s\).

Calculate the predictor-distance contribution.
Find \(x_0-\bar{x}\), square it, and divide by \(S_{xx}\).

Add the sample-size contribution.
Add \(1/n\) to the result from Step 2.

Multiply by \(s\) and take the square root.
The result is in the response units.

Worked Calculations

Worked Example: A Target at the Predictor Mean

A hypothetical insurer studies the relationship between a branch’s weekly training hours and its mean processing time per claim. The regression uses \(n=8\) branches, with \(\bar{x}=10\) training hours, \(S_{xx}=56\) hours squared, and residual standard deviation \(s=3\) minutes. Find the standard error of the fitted mean processing time at \(x_0=10\). Assume fixed predictors and independent normal errors with mean zero and common variance \(\sigma^2\). No interest-rate or mortality assumption is involved.

Method. Apply the fitted-mean standard error formula. Because \(x_0=\bar{x}\), the predictor-distance term is zero.

\begin{aligned} SE\{\hat{y}(10)\} &=3\sqrt{\frac{1}{8}+\frac{(10-10)^2}{56}}\\ &=3\sqrt{\frac{1}{8}+0}\\ &=3\sqrt{\frac{1}{8}}\\ &=\frac{3}{\sqrt{8}}\\ &\approx 1.0607\text{ minutes}. \end{aligned}

The fitted mean’s standard error is approximately 1.0607 minutes. The predictor mean eliminates the distance contribution, but it does not eliminate the \(1/n\) contribution. The common mistake of dropping \(1/n\) would incorrectly give a standard error of zero.

Worked Example: A Target Away From the Predictor Mean

A hypothetical study models the mean monthly number of service requests per policy as a function of an office’s training score. For \(n=10\) offices, the summaries are \(\bar{x}=5\), \(S_{xx}=90\), and \(s=4\) requests. Calculate the standard error of the fitted mean at \(x_0=8\). Assume fixed predictor values and independent normal errors with mean zero and common variance \(\sigma^2\). Interest and mortality assumptions do not apply to this regression calculation.

Method. Use the squared distance from the predictor mean, scaled by \(S_{xx}\), in addition to \(1/n\).

\begin{aligned} x_0-\bar{x}&=8-5=3,\\ (x_0-\bar{x})^2&=3^2=9,\\ \frac{(x_0-\bar{x})^2}{S_{xx}}&=\frac{9}{90}=0.1,\\ \frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}} &=\frac{1}{10}+0.1=0.2,\\ SE\{\hat{y}(8)\}&=4\sqrt{0.2}=\sqrt{3.2} \approx 1.7889\text{ requests}. \end{aligned}

The standard error is approximately 1.7889 requests. The distance from \(\bar{x}\) adds uncertainty beyond the sample-mean contribution. The calculation avoids the mistake of using only \(s/\sqrt{n}\), which applies at \(x_0=\bar{x}\), not generally at an arbitrary target value.

Worked Example: Calculating the Standard Error From SSE

A hypothetical regression relates a clinic’s weekly outreach hours to its mean weekly follow-up count. There are \(n=6\) clinics, \(SSE=32\), \(\bar{x}=4\) hours, and \(S_{xx}=20\) hours squared. Find the standard error of the fitted mean at \(x_0=6\). Assume fixed predictors and independent normal errors with mean zero and common variance \(\sigma^2\). No interest rate or mortality assumption is relevant.

Method. First estimate the residual variance using the regression residual degrees of freedom, then calculate the fitted-mean standard error.

\begin{aligned} n-2&=6-2=4,\\ MSE&=\frac{SSE}{n-2}=\frac{32}{4}=8,\\ s&=\sqrt{8},\\ x_0-\bar{x}&=6-4=2,\\ \frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}} &=\frac{1}{6}+\frac{2^2}{20}\\ &=\frac{1}{6}+\frac{4}{20}\\ &=\frac{1}{6}+\frac{1}{5} =\frac{5}{30}+\frac{6}{30} =\frac{11}{30},\\ SE\{\hat{y}(6)\} &=\sqrt{8}\sqrt{\frac{11}{30}} =\sqrt{\frac{88}{30}} =\sqrt{\frac{44}{15}} \approx 1.7127. \end{aligned}

The standard error is approximately 1.7127 follow-ups per week. The degrees of freedom are \(n-2\), not \(n-1\), because both the intercept and slope were estimated. Using \(SSE/(n-1)\) would give the wrong residual variance and hence the wrong standard error.

How Sample Size and Predictor Spread Affect Precision

At \(x_0=\bar{x}\), the formula simplifies to \(s/\sqrt{n}\). Holding \(s\) fixed, a larger sample reduces this central standard error. Away from \(\bar{x}\), the extra contribution depends on \((x_0-\bar{x})^2/S_{xx}\). Increasing predictor spread, while keeping the target’s distance and other quantities fixed, reduces this contribution. A target farther from the predictor mean increases it.

These comparisons are conditional on the other quantities being held fixed. In an actual study, changing the sample size or the range of predictors may also change \(s\), \(\bar{x}\), and the fitted model. The formula describes how the components contribute; it does not guarantee that every change in a real data set improves precision.

The standard error is expressed in the units of the response. For example, if \(Y\) is measured in dollars, then \(s\) and the standard error are in dollars; \(S_{xx}\) is in squared predictor units, so the quantity inside the square root is dimensionless.

Common Mistakes and a Useful Check

Omitting \(1/n\). At the predictor mean the distance term is zero, but the standard error is \(s/\sqrt{n}\), not zero.
Using the wrong residual degrees of freedom. In simple linear regression, \(MSE=SSE/(n-2)\), because two regression coefficients are estimated.
Forgetting to square the distance. Use \((x_0-\bar{x})^2\); targets equally far above and below the mean must give the same standard error.
Confusing \(s\) with the fitted-mean standard error. \(s\) estimates the residual standard deviation; the fitted-mean standard error also depends on \(n\), \(x_0\), and \(S_{xx}\).
Adding variability for a future response. This formula is for the fitted mean response, not one future observation. A future response includes additional individual-response variability.

A quick check is that the expression inside the square root must be positive and at least \(1/n\). Therefore the standard error cannot be less than \(s/\sqrt{n}\) when \(s>0\); equality occurs at \(x_0=\bar{x}\). A computed value below that minimum signals an arithmetic or formula error.

Key takeaway: The standard error of the fitted mean at \(x_0\) is \(s\sqrt{1/n+(x_0-\bar{x})^2/S_{xx}}\). The sample-size term remains at every target value; distance from the predictor mean adds uncertainty, while greater predictor spread reduces that distance contribution.

Check Your Understanding

For each problem, use the fitted-mean standard error formula and retain unrounded values until the final answer. Round standard errors to four decimal places.

A regression has \(n=16\), \(s=3\), and \(x_0=\bar{x}\). Find the standard error. (Answer: 0.7500)
A fitted mean is required at \(x_0=9\), where \(n=9\), \(\bar{x}=5\), \(S_{xx}=32\), and \(s=2\). Find its standard error. (Answer: 1.5635)
A regression has \(n=7\), \(SSE=45\), \(\bar{x}=12\), \(S_{xx}=36\), and target \(x_0=15\). Find the standard error, first calculating \(s\) from \(SSE\). (Answer: 1.8804)
For \(n=12\), \(s=5\), \(\bar{x}=20\), and \(S_{xx}=220\), find the standard error at \(x_0=24\). (Answer: 1.9752)
A regression has \(n=10\), \(s=2\), \(\bar{x}=6\), and \(S_{xx}=45\). Compare the standard errors at \(x_0=6\) and \(x_0=3\). (Answer: At \(x_0=6\), 0.6325; at \(x_0=3\), 1.0954.)

What You'll Learn