Standard Errors For Slope And Intercept

Why Regression Coefficients Have Standard Errors

A fitted regression line describes a sample, but a different sample of responses could produce a different line. The estimated standard error of a coefficient measures the sampling variability expected in that estimate under a specified regression model. It is therefore not the size of a residual, nor the uncertainty in an individual prediction. This tutorial develops the standard errors for the slope and intercept in simple linear regression.

The formulas use two ingredients: an estimate of the error variance, calculated from the residuals, and the arrangement of the predictor values. More spread in the predictor values generally makes the slope easier to estimate. The intercept also depends on where the predictor values are centered relative to zero.

Model and assumptions: For observations $i=1,\ldots,n$, assume the simple linear regression model $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$, where the predictor values $x_i$ are treated as fixed, and the errors $\varepsilon_i$ are independent with mean zero and common variance $\sigma^2$. Require $n\geq 3$ and at least two distinct predictor values. Normality is not needed to calculate the standard error formulas; it is an additional assumption for exact small-sample inference.

Write the fitted line as $\hat{Y}_i=\hat{\beta}_0+\hat{\beta}_1x_i$, and define the residual $e_i=Y_i-\hat{Y}_i$. The predictor spread is summarized by $S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2$, where $\bar{x}=\sum x_i/n$. The residual sum of squares is $\mathrm{SSE}=\sum e_i^2$.

Definition: In simple linear regression with an intercept, the residual mean square is $\mathrm{MSE}=\mathrm{SSE}/(n-2)$. The estimated standard errors of the least-squares slope and intercept are $$ \widehat{\mathrm{SE}}(\hat{\beta}_1)=\sqrt{\frac{\mathrm{MSE}}{S_{xx}}} \qquad\text{and}\qquad \widehat{\mathrm{SE}}(\hat{\beta}_0)=\sqrt{\mathrm{MSE}\left(\frac{1}{n}+\frac{\bar{x}^{\,2}}{S_{xx}}\right)}. $$

The quantity $n-2$ appears because fitting an intercept and a slope uses two degrees of freedom. The earlier material on residual variance and degrees of freedom establishes this residual mean square as an estimate of $\sigma^2$. Substituting it for the unknown error variance in the coefficient variance formulas gives the estimated standard errors.

Why These Formulas Hold

A short derivation shows how predictor spread enters the slope formula and why the intercept formula includes both $1/n$ and $\bar{x}^{\,2}/S_{xx}$. Conditional on the fixed predictor values, the least-squares slope can be written as $$ \hat{\beta}_1=\beta_1+ \frac{\sum_{i=1}^n(x_i-\bar{x})\varepsilon_i}{S_{xx}}. $$

The errors have variance $\sigma^2$ and are independent, so the variance of the weighted sum in the numerator is $\sigma^2\sum(x_i-\bar{x})^2=\sigma^2S_{xx}$. Dividing by $S_{xx}^2$ gives $$ \operatorname{Var}(\hat{\beta}_1\mid x_1,\ldots,x_n)=\frac{\sigma^2}{S_{xx}}. $$

The fitted line passes through $(\bar{x},\bar{Y})$, so $\hat{\beta}_0=\bar{Y}-\hat{\beta}_1\bar{x}$. The sample mean response has variance $\sigma^2/n$, and its covariance with the slope estimate is zero because the centered predictor values sum to zero. Therefore, $$ \operatorname{Var}(\hat{\beta}_0\mid x_1,\ldots,x_n) =\frac{\sigma^2}{n}+\bar{x}^{\,2}\frac{\sigma^2}{S_{xx}} =\sigma^2\left(\frac{1}{n}+\frac{\bar{x}^{\,2}}{S_{xx}}\right). $$

Replacing $\sigma^2$ by $\mathrm{MSE}$ and taking square roots produces the estimated standard errors. This derivation also makes the roles of the predictor values visible: larger $S_{xx}$ reduces slope variance, while an intercept estimated far from the center of the observed predictor values can be less precise.

Calculating Both Standard Errors

Worked Example: A Small Regression with Residuals Given

A hypothetical analyst records a response $Y$ for four accounts at predictor values $x=1,2,3,4$. The fitted line is $\hat{Y}=3+2x$, and the residuals in the same order are $1,-1,-1,1$. Calculate the estimated standard errors of the slope and intercept. State the method and a common mistake to avoid.

Step 1: Calculate the residual sum of squares. Squaring each residual gives $1^2=1$, $(-1)^2=1$, $(-1)^2=1$, and $1^2=1$. Thus $\mathrm{SSE}=1+1+1+1=4$.

Step 2: Find the residual mean square. Here $n=4$, so the residual degrees of freedom are $4-2=2$. Therefore $\mathrm{MSE}=4/2=2$.

Step 3: Calculate the predictor summaries. The mean is $\bar{x}=(1+2+3+4)/4=2.5$. The centered values are $-1.5,-0.5,0.5,1.5$, so $S_{xx}=2.25+0.25+0.25+2.25=5$.

Step 4: Calculate the slope standard error. $$ \widehat{\mathrm{SE}}(\hat{\beta}_1)=\sqrt{\frac{2}{5}} =\sqrt{0.4}=0.6325. $$

Step 5: Calculate the intercept standard error. The quantity inside the parentheses is $1/4+(2.5)^2/5=0.25+6.25/5=0.25+1.25=1.5$. Hence $$ \widehat{\mathrm{SE}}(\hat{\beta}_0)=\sqrt{2(1.5)} =\sqrt{3}=1.7321. $$

Check and method. The residuals sum to zero, and their products with the centered predictor values sum to $(-1.5)(1)+(-0.5)(-1)+(0.5)(-1)+(1.5)(1)=0$, as expected for this least-squares fit with an intercept. The method is to calculate MSE and $S_{xx}$, then substitute into each formula. A common mistake is dividing SSE by $n-1$; that would give the wrong residual variance estimate because two coefficients, not one, were fitted.

Worked Example: Using Regression Output to Find Standard Errors

A hypothetical study relates a measurement $Y$ to a predictor $x$. The five predictor values are $2,4,6,8,10$, and the observed responses are $18,20,28,32,42$. Find the estimated standard errors of the slope and intercept. Give the standard errors to four decimal places, and identify the formula trap this calculation illustrates.

Step 1: Verify the fitted line and residuals. The fitted line is $\hat{Y}=10+3x$, giving fitted values $16,22,28,34,40$. Subtracting these from the observed responses gives residuals $2,-2,0,-2,2$. Their sum is zero, and their products with the centered predictor values $-4,-2,0,2,4$ sum to $-8+4+0-4+8=0$.

Step 2: Calculate SSE and MSE. The squared residuals are $4,4,0,4,4$, so $\mathrm{SSE}=16$. With $n=5$, the residual degrees of freedom are $5-2=3$, and $\mathrm{MSE}=16/3=5.3333\ldots$.

Step 3: Calculate $\bar{x}$ and $S_{xx}$. The predictor mean is $\bar{x}=(2+4+6+8+10)/5=6$. The squared centered values sum to $16+4+0+4+16=40$, so $S_{xx}=40$.

Step 4: Calculate the slope standard error. $$ \widehat{\mathrm{SE}}(\hat{\beta}_1) =\sqrt{\frac{16/3}{40}} =\sqrt{\frac{2}{15}} =0.3651. $$

Step 5: Calculate the intercept standard error. The factor for the intercept is $1/5+6^2/40=0.2+36/40=1.1$. Therefore, $$ \widehat{\mathrm{SE}}(\hat{\beta}_0) =\sqrt{\frac{16}{3}(1.1)} =\sqrt{\frac{88}{15}} =2.4221. $$

Check and method. The fitted values differ from the observed responses by exactly the listed residuals, whose squares total 16. The method is direct substitution after verifying the regression summaries. The trap is omitting the square on $\bar{x}$ in the intercept formula: using $1/n+\bar{x}/S_{xx}$ is incorrect. The required term is $1/n+\bar{x}^{\,2}/S_{xx}$.

Predictor Spread and Location

Worked Example: Comparing Two Predictor Designs

Two hypothetical studies each have $n=5$, predictor mean $\bar{x}=10$, and residual sum of squares $\mathrm{SSE}=27$. In Study A, $S_{xx}=20$; in Study B, $S_{xx}=80$. Compare the estimated standard errors of the slope and intercept. Round to four decimal places, and explain what changes when the predictor values are more spread out.

Step 1: Find the shared residual mean square. Each study has $5-2=3$ residual degrees of freedom. Thus $\mathrm{MSE}=27/3=9$ in both studies.

Step 2: Calculate slope standard errors. For Study A, $\widehat{\mathrm{SE}}(\hat{\beta}_1)=\sqrt{9/20}=\sqrt{0.45}=0.6708$. For Study B, $\widehat{\mathrm{SE}}(\hat{\beta}_1)=\sqrt{9/80}=\sqrt{0.1125}=0.3354$.

Step 3: Calculate intercept standard errors. For Study A, the intercept factor is $1/5+10^2/20=0.2+5=5.2$, giving $\sqrt{9(5.2)}=\sqrt{46.8}=6.8411$. For Study B, the factor is $0.2+100/80=1.45$, giving $\sqrt{9(1.45)}=\sqrt{13.05}=3.6125$.

Check and interpretation. Both studies have the same sample size, predictor mean, and residual mean square. The fourfold increase in $S_{xx}$ in Study B halves the slope standard error, because the slope variance is proportional to $1/S_{xx}$. It also reduces the intercept standard error here, because the predictor values provide more information about the line near its center. The method is to hold the other summaries fixed and compare each formula. A common mistake is to interpret the slope standard error as a summary of uncertainty in the entire fitted line; the intercept has its own standard error and depends on $\bar{x}$ as well as $S_{xx}$.

Interpreting and Using the Estimates

An estimated standard error is expressed in the same units as the coefficient it describes. If the response is measured in dollars and the predictor in years, the slope and its standard error are measured in dollars per year; the intercept and its standard error are measured in dollars. The standard error summarizes the estimated sampling spread of a coefficient across repeated samples under the model. It is not a guarantee that a particular estimate is close to the unknown coefficient.

The formulas also make clear why two regression analyses with the same residual mean square can have different coefficient precision. More observations tend to reduce uncertainty, and a wider range of predictor values increases $S_{xx}$, improving slope precision. For the intercept, $1/n$ reflects the contribution from estimating the mean response, while $\bar{x}^{\,2}/S_{xx}$ reflects how far the predictor center is from zero relative to its spread. If zero lies far outside the observed predictor range, the intercept may be an extrapolation and its standard error can be large.

These standard-error formulas rely on independent errors with a common variance and the stated linear mean model. If error variances differ across observations, or errors are dependent, the displayed formulas generally do not give valid standard errors without adjustment. The formulas also require $S_{xx}>0$; if every predictor value is identical, a slope cannot be estimated separately from the intercept. A zero MSE from a perfect fit produces zero estimated standard errors by these formulas, but it does not establish that the model assumptions are appropriate.

Standard errors are the scale quantities used in later regression inference, but they are not themselves tests or confidence intervals. For this calculation, keep the task focused: establish the model conditions, compute $\mathrm{MSE}$, $\bar{x}$, and $S_{xx}$, and apply the correct coefficient-specific formula.

Key takeaway: In simple linear regression with an intercept, estimate the error variance by $\mathrm{MSE}=\mathrm{SSE}/(n-2)$. Divide MSE by $S_{xx}$ for the slope variance estimate; multiply it by $1/n+\bar{x}^{\,2}/S_{xx}$ for the intercept variance estimate. Take square roots to obtain the estimated standard errors.

Check Your Understanding

For each problem, calculate the requested standard errors using the simple linear regression formulas. Round numerical standard errors to four decimal places.

A regression has $n=6$, $\bar{x}=4$, $S_{xx}=15$, and $\mathrm{SSE}=20$. Find the estimated standard errors of the slope and intercept. (Answer: $0.5774$ for the slope and $2.4833$ for the intercept.)
A regression has $n=8$, $\bar{x}=0$, $S_{xx}=32$, and $\mathrm{MSE}=4$. Find both estimated standard errors and explain why the intercept formula simplifies. (Answer: $0.3536$ for the slope and $0.7071$ for the intercept; the $\bar{x}^{\,2}/S_{xx}$ term is zero.)
A regression has $n=5$, $\bar{x}=3$, $S_{xx}=12$, and $\mathrm{SSE}=18$. Calculate both estimated standard errors, first finding the residual degrees of freedom. (Answer: $\mathrm{MSE}=6$, slope standard error $0.7071$, and intercept standard error $2.3875$.)
A regression has $n=10$, $\bar{x}=5$, $S_{xx}=45$, and $\mathrm{SSE}=56$. A candidate divides SSE by $n-1$ before calculating the standard errors. Give the correct standard errors and identify the correct denominator. (Answer: use $n-2=8$, so $\mathrm{MSE}=7$; slope standard error $0.3944$ and intercept standard error $2.1422$.)
A regression has $n=12$, $\bar{x}=-2$, $S_{xx}=50$, and $\mathrm{MSE}=2.5$. Calculate both estimated standard errors. (Answer: slope standard error $0.2236$ and intercept standard error $0.6390$.)

What You'll Learn