Why the Error-Variance Estimate Uses n Minus 2

Why the Divisor Depends on Two Estimated Coefficients

In simple linear regression, the residual sum of squares measures the variation left after fitting a line to the observed responses. The usual estimate of the error variance divides this sum by $n-2$, not by $n$. The reason is not a rule to memorize: fitting both an intercept and a slope uses two pieces of information from the data. Under the fixed-predictor model, we can derive exactly how that affects the expected residual sum of squares.

The conclusion depends on the assumptions. In particular, it concerns independent errors with mean zero and a common variance, with the predictor values treated as fixed. Normality of the errors is not needed for this expectation result.

Model and assumptions: For fixed predictor values $x_1,\ldots,x_n$, suppose $$ Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad i=1,\ldots,n, $$ where $\beta_0$ and $\beta_1$ are fixed constants. Assume the errors are independent, $E[\varepsilon_i]=0$, and $\operatorname{Var}(\varepsilon_i)=\sigma^2$ for every $i$. Define $\bar{x}=n^{-1}\sum_{i=1}^n x_i$ and $S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2$. The least-squares slope is identifiable when $S_{xx}>0$, meaning the predictor values are not all identical.

The Residual Sum of Squares and Its Expectation

Let $\hat{\beta}_0$ and $\hat{\beta}_1$ be the least-squares intercept and slope. For each observation, the fitted value is $\hat{Y}_i=\hat{\beta}_0+\hat{\beta}_1x_i$, and the residual is $e_i=Y_i-\hat{Y}_i$. The residual sum of squares is

\mathrm{SSE}=\sum_{i=1}^n e_i^2.

Here $\mathrm{SSE}$ is a quantity calculated from the observed sample; it can vary from one sample to another. Its expectation, $E[\mathrm{SSE}]$, describes its average over repeated samples generated under the stated model, with the predictor values held fixed.

Theorem: Under the stated fixed-predictor model, if $S_{xx}>0$, then $$ E[\mathrm{SSE}]=(n-2)\sigma^2. $$ When $n>2$, the estimator $s^2=\mathrm{SSE}/(n-2)$ is therefore unbiased for $\sigma^2$. When $n=2$, the expectation identity remains true, but division by $n-2$ is undefined, so this formula does not provide an estimator.

Deriving the Expectation

The least-squares coefficient formulas can be written in terms of the errors. Set $c_i=x_i-\bar{x}$, so that $\sum_i c_i=0$ and $\sum_i c_i^2=S_{xx}$. Also define $\bar{\varepsilon}=n^{-1}\sum_i\varepsilon_i$ and $A=\sum_i c_i\varepsilon_i$. Substituting the model into the slope formula gives

\hat{\beta}_1 =\frac{\sum_i c_i(Y_i-\bar{Y})}{S_{xx}} =\beta_1+\frac{A}{S_{xx}}.

The intercept formula $\hat{\beta}_0=\bar{Y}-\hat{\beta}_1\bar{x}$ then gives $\hat{\beta}_0=\beta_0+\bar{\varepsilon}-\bar{x}A/S_{xx}$. Thus the residual for observation $i$ is

e_i=\varepsilon_i-\bar{\varepsilon}-\frac{c_iA}{S_{xx}}.

This expresses the fitted residuals in terms of the original errors and the two fitted coefficients. To find the sum of their squares, first let $q_i=\varepsilon_i-\bar{\varepsilon}$. Since $\sum_i c_i=0$, we have $\sum_i q_i=0$ and $\sum_i c_iq_i=A$. Expanding the squared residuals now yields

\begin{aligned} \mathrm{SSE} &=\sum_i\left(q_i-\frac{c_iA}{S_{xx}}\right)^2\\ &=\sum_iq_i^2-\frac{2A}{S_{xx}}\sum_i c_iq_i +\frac{A^2}{S_{xx}^2}\sum_i c_i^2\\ &=\sum_iq_i^2-\frac{A^2}{S_{xx}}\\ &=\sum_i\varepsilon_i^2-\frac{\left(\sum_i\varepsilon_i\right)^2}{n} -\frac{A^2}{S_{xx}}. \end{aligned}

For the last equality, expanding $\sum_i(\varepsilon_i-\bar{\varepsilon})^2$ gives $\sum_i\varepsilon_i^2-n\bar{\varepsilon}^2=\sum_i\varepsilon_i^2-(\sum_i\varepsilon_i)^2/n$. Taking expectations term by term completes the argument. The independence and zero-mean assumptions imply $E[(\sum_i\varepsilon_i)^2]=n\sigma^2$. They also give $E[A^2]=\sigma^2\sum_i c_i^2=\sigma^2S_{xx}$, since the cross-products between distinct errors have expectation zero. Finally, $E[\sum_i\varepsilon_i^2]=n\sigma^2$. Therefore

E[\mathrm{SSE}] =n\sigma^2-\frac{n\sigma^2}{n} -\frac{\sigma^2S_{xx}}{S_{xx}} =(n-2)\sigma^2.

The two subtractions each equal $\sigma^2$: one comes from fitting the intercept, and the other from fitting the slope. This is why the result has $n-2$, rather than $n$, multiplying the error variance.

Worked Applications

Worked Example: Calculating the Error-Variance Estimate

A hypothetical study records five responses at fixed predictor values $x=0,1,2,3,4$, obtaining $Y=9,14,12,18,17$, respectively. Under the fixed-predictor regression model with independent zero-mean errors of common variance, calculate the least-squares residual sum of squares and the unbiased estimate of the error variance.

Method. Fit the least-squares line, calculate its residuals, and divide the residual sum of squares by $n-2$. The predictor values are not all identical, and $n=5>2$, so the estimate is defined.

The predictor mean is $\bar{x}=(0+1+2+3+4)/5=2$, and $S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10$. The response mean is $\bar{Y}=(9+14+12+18+17)/5=70/5=14$. The slope is \[ \hat{\beta}_1 =\frac{(-2)(9-14)+(-1)(14-14)+0(12-14)+1(18-14)+2(17-14)}{10} =\frac{10+0+0+4+6}{10}=2. \] The intercept is $\hat{\beta}_0=\bar{Y}-\hat{\beta}_1\bar{x}=14-2(2)=10$, so the fitted line is $\hat{Y}=10+2x$.

The fitted values are $10,12,14,16,18$, giving residuals $-1,2,-2,2,-1$. Hence $\mathrm{SSE}=(-1)^2+2^2+(-2)^2+2^2+(-1)^2=1+4+4+4+1=14$. The estimate is $s^2=14/(5-2)=14/3=4.6667$, rounded to four decimal places. Its units are response-units squared.

Check. The residuals sum to $-1+2-2+2-1=0$, and their products with the centered predictors sum to $2-2+0+2-2=0$, consistent with the fitted least-squares line. The common mistake avoided here is dividing by $n$ instead of $n-2$.

Worked Example: Checking the Expected SSE from the Model

A hypothetical regression uses the fixed predictor values $2,4,6,8$. Its errors are independent with mean zero and variance $\sigma^2=9$. Calculate the expected residual sum of squares by identifying the two reductions in the derivation.

Here $n=4$, $\bar{x}=(2+4+6+8)/4=5$, and the centered predictors are $-3,-1,1,3$. Thus $S_{xx}=9+1+1+9=20$.

The expected sum of squared errors before fitting is $E[\sum_i\varepsilon_i^2]=4(9)=36$. The intercept-related term has expectation $E[(\sum_i\varepsilon_i)^2]/n=(4\cdot9)/4=9$. For the slope-related term, $E[A^2]/S_{xx}=(9\cdot20)/20=9$. Therefore $E[\mathrm{SSE}]=36-9-9=18$.

The theorem gives the same result: $(n-2)\sigma^2=(4-2)(9)=18$. The calculation checks that each fitted coefficient accounts for one $\sigma^2$ in the expected sum of squares.

Worked Example: Why Two Observations Do Not Give This Estimate

A hypothetical data set has two distinct fixed predictor values, $x_1=1$ and $x_2=3$, and observed responses $Y_1=7$ and $Y_2=11$. Find the least-squares line and determine whether $\mathrm{SSE}/(n-2)$ estimates the error variance.

With two distinct predictor values, the line through the two observations has slope $\hat{\beta}_1=(11-7)/(3-1)=4/2=2$, and intercept $\hat{\beta}_0=7-2(1)=5$. It predicts $7$ at $x=1$ and $11$ at $x=3$, so both residuals are zero and $\mathrm{SSE}=0^2+0^2=0$.

Here $n-2=0$, so $\mathrm{SSE}/(n-2)=0/0$ is undefined, not an error-variance estimate. The expectation identity still holds: $E[\mathrm{SSE}]=(2-2)\sigma^2=0$. The mistake avoided is treating an identity valid for the expected SSE as permission to divide by zero.

Worked Example: The Bias from Using the Wrong Divisor

Suppose $n=5$, the fixed predictor values are not all equal, and the independent errors have mean zero and common variance $\sigma^2$. A candidate proposes estimating the error variance with $\mathrm{SSE}/(n-1)$. Determine the expectation of this proposed estimate and its bias.

The model result gives $E[\mathrm{SSE}]=(5-2)\sigma^2=3\sigma^2$. Since $n-1=4$, the proposed estimate has expectation $E[\mathrm{SSE}/4]=(3/4)\sigma^2$. Its bias is its expectation minus the target: $(3/4)\sigma^2-\sigma^2=-\sigma^2/4$. Thus it is biased downward unless $\sigma^2=0$. Using the correct divisor $n-2=3$ instead gives $E[\mathrm{SSE}/3]=(3\sigma^2)/3=\sigma^2$. The trap is substituting the familiar sample-variance divisor $n-1$ without accounting for the two fitted regression coefficients.

When the Result Applies—and When It Does Not

The derivation requires fixed predictors, independent errors, zero error means, a common error variance, and a design with $S_{xx}>0$. If the errors have unequal variances or are dependent, the calculations of $E[(\sum_i\varepsilon_i)^2]$ and $E[A^2]$ generally change. The simple $(n-2)\sigma^2$ result need not hold. If the conditional error means are not zero, the fitted line is not being analyzed under the stated model either.

Normal errors are not an assumption needed for this expectation calculation. Adding a normality assumption can support other distributional results, but it is not required to show that $\mathrm{SSE}/(n-2)$ is unbiased when $n>2$ and the stated model holds. Also distinguish unbiasedness from the value in one data set: an unbiased estimate need not equal the unknown $\sigma^2$ for a particular sample.

Key takeaway: With an intercept and an identifiable slope, independent zero-mean errors of common variance imply $E[\mathrm{SSE}]=(n-2)\sigma^2$. Dividing by $n-2$ gives an unbiased error-variance estimate only when $n>2$; at $n=2$, the expectation identity remains true but the quotient is undefined.

Check Your Understanding

Use the fixed-predictor assumptions stated above unless a question specifies otherwise.

A regression has $n=7$, distinct predictor values, and observed $\mathrm{SSE}=30$. Calculate the unbiased estimate of the error variance. (Answer: $30/(7-2)=6$)
For a model with $n=6$, fixed nonconstant predictors, and independent errors with variance $4$, find $E[\mathrm{SSE}]$. (Answer: $(6-2)(4)=16$)
A regression has two observations at distinct predictor values and includes an intercept and slope. What is its residual sum of squares, and is $\mathrm{SSE}/(n-2)$ defined? (Answer: $\mathrm{SSE}=0$; the quotient is undefined because $n-2=0$)
For $n=8$ and nonconstant fixed predictors, a model implies $E[\mathrm{SSE}]=42$. Find the error variance under the stated assumptions. (Answer: $\sigma^2=42/(8-2)=7$)
For $n=5$, someone uses $\mathrm{SSE}/4$ to estimate $\sigma^2$. What is the expectation of this quantity and its bias? (Answer: $E[\mathrm{SSE}/4]=3\sigma^2/4$; bias $=-\sigma^2/4$)

What You'll Learn