Standard Errors of the OLS Coefficients

Why Coefficient Standard Errors Matter

An estimated regression coefficient is calculated from a particular sample. A different sample of responses, even at the same predictor values, would generally produce a different estimated slope and intercept. A coefficient’s standard error measures the estimated sampling variability of that coefficient: it is an estimate of the standard deviation of the coefficient estimator across repeated samples under the model.

In simple linear regression, the standard error of the slope reflects how precisely the data determine the change in the mean response per unit increase in the predictor. The intercept standard error reflects the precision of the fitted mean response at \(x=0\). The two standard errors depend on the error variation and on the arrangement of the predictor values, but in different ways.

Formulas for the Two Standard Errors

Write \(\bar{x}=\sum_{i=1}^n x_i/n\) and \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\). Provided \(S_{xx}>0\), the ordinary least-squares (OLS) slope and intercept estimates are

The coefficient variances involve the unknown \(\sigma^2\). In practice, estimate it using the residual sum of squares \(\mathrm{SSE}=\sum_{i=1}^n(Y_i-b_0-b_1x_i)^2\). The residual variance estimate is \(s^2=\mathrm{SSE}/(n-2)\), and the residual standard error is \(s=\sqrt{s^2}\). The two degrees of freedom are used because the fitted line estimates two coefficients.

Here \(\operatorname{Var}(b_0)\) and \(\operatorname{Var}(b_1)\) are the theoretical variances under the model, while the standard errors replace the unknown \(\sigma\) with its estimate \(s\). They are not the same as the residual standard error: \(s\) estimates the scale of the observation-level errors, whereas each coefficient standard error describes the estimated variability of a particular coefficient.

Why Predictor Spread and Location Matter

The slope formula contains \(S_{xx}\) in the denominator. When the predictor values are more spread out around their mean, \(S_{xx}\) is larger, so the slope standard error is smaller if the residual standard error remains unchanged. A wider range of predictor values provides more information about the rate of change in the fitted line.

The intercept standard error has two components. The \(1/n\) component reflects the contribution of estimating an average response. The additional \(\bar{x}^2/S_{xx}\) component reflects how far \(x=0\) is from the center of the observed predictors relative to their spread. If \(\bar{x}=0\), the observed predictor values are centered at zero and the intercept standard error is \(s/\sqrt{n}\). If zero is far from \(\bar{x}\), the standard error is larger. This is because the intercept is the fitted response at \(x=0\), which may be far from the center of the data.

The formulas follow directly by expressing each estimator in terms of the errors. Since \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), the slope estimator can be written as

The errors have common variance \(\sigma^2\) and zero covariances, so the variance of this weighted sum is the sum of the squared weights times \(\sigma^2\). Thus

For the intercept, use \(b_0=\bar{Y}-b_1\bar{x}\). Substituting the model into \(\bar{Y}\) gives

The coefficient of \(\varepsilon_i\) in this expression is \(1/n-\bar{x}(x_i-\bar{x})/S_{xx}\). Squaring and summing these coefficients, then multiplying by \(\sigma^2\), gives the variance. The cross-product term is zero because \(\sum_{i=1}^n(x_i-\bar{x})=0\); the remaining terms are

Replacing \(\sigma\) by \(s\) in the square roots gives the estimated standard errors.

Common Mistake: Treating the Intercept Like the Slope

A frequent error is to use \(s/\sqrt{n}\) as the intercept standard error in every regression. That formula applies to the intercept only when \(\bar{x}=0\). In the first worked example, this incorrect calculation would give

But the predictor mean is 10, not zero. The correct calculation includes the location term:

The incorrect result is too small because it omits the contribution from estimating the fitted response at \(x=0\), away from the center of these predictor values. A second common error is using \(n\), rather than \(n-2\), to estimate the residual variance. Always calculate \(s^2=\mathrm{SSE}/(n-2)\) before calculating either coefficient standard error.

Interpreting the Formulas Carefully

The formulas quantify precision under the stated model; they do not establish that the model is suitable for a particular data set. They also do not, by themselves, provide a probability statement about a coefficient. For example, calculating \(\operatorname{SE}(b_1)\) does not imply that a fixed proportion of future estimates will lie within one standard error of \(\beta_1\). Such claims require additional distributional results and conditions.

When comparing coefficient precision, check all relevant quantities. More observations can reduce standard errors, but the formulas also depend on residual variation and predictor spread. For the intercept, the predictor mean matters as well as its spread. Thus, a large slope standard error may reflect a small \(S_{xx}\), a large \(s\), or both; a large intercept standard error may additionally reflect a predictor center far from zero.