Why Estimate Error Variance?
A regression model describes a response as a systematic component, determined by predictors, plus random error. Even after estimating the regression coefficients, the errors are not observed directly. The fitted residuals provide the information used to estimate how much unexplained variation remains. The key calculation is the residual mean square, which divides the residual sum of squares by the residual degrees of freedom.
The divisor is not simply the number of observations. Estimating the coefficients imposes constraints on the residuals: in ordinary least squares, the residual vector must be perpendicular to the columns of the design matrix. The residual degrees of freedom count the remaining independent directions in which residual variation can occur.
The Regression Assumptions and the Variance Estimate
For the unbiasedness result, treat the design matrix \(X\) as fixed and assume the model is correctly specified: \(E[\varepsilon\mid X]=0\) and \(\operatorname{Var}(\varepsilon\mid X)=\sigma^2 I\), where \(\sigma^2\) is the common error variance. These assumptions mean that the errors have mean zero, equal variance, and no pairwise correlation, conditional on the predictors. Normality is not required for \(MSE\) to be unbiased.
If the errors are also independent and normally distributed, then \(SSE/\sigma^2\) has a chi-square distribution with \(n-r\) degrees of freedom. This distributional result is more restrictive than unbiasedness: it requires normal errors. If error variances differ or errors are correlated, the ordinary residual mean square need not estimate a single common error variance without bias.
When the design matrix has full column rank, \(r=p\), so the denominator is \(n-p\). If the design matrix is rank deficient, the number of columns can exceed its rank; the correct residual degrees of freedom are \(n-r\), not \(n\) minus the raw number of columns. The earlier material on rank deficiency explains why a rank-deficient design does not identify all coefficients separately.
Why the Degrees of Freedom Are \(n-r\)
The fitted values are projections of the response vector onto the column space of \(X\). Ordinary least squares chooses the projection that minimizes the sum of squared residuals. The residual vector therefore lies in the orthogonal complement of that column space. Since the column space has dimension \(r\) inside an \(n\)-dimensional observation space, its orthogonal complement has dimension \(n-r\). That is the geometric reason for the residual degrees of freedom.
The same result can be shown algebraically, and the algebra also establishes why dividing by \(n-r\) gives an unbiased estimate. Assume \(X\) has full column rank for the displayed inverse. Define the projection matrix \(H=X(X'X)^{-1}X'\) and the residual-maker matrix \(M=I-H\). Then \(\hat{Y}=HY\) and \(e=MY\). Because \(MX=0\), substituting \(Y=X\beta+\varepsilon\) gives \(e=M\varepsilon\). The matrix \(M\) is symmetric and idempotent, and its rank and trace are both \(n-r\).
Since \(e=M\varepsilon\) and \(M\) is symmetric and idempotent, \(SSE=e'e=\varepsilon'M\varepsilon\).
For a mean-zero error vector with covariance matrix \(\sigma^2 I\), the expectation of this quadratic form is \(\sigma^2\operatorname{tr}(M)\).
\(\operatorname{tr}(M)=\operatorname{rank}(M)=n-r\), so \(E[SSE\mid X]=(n-r)\sigma^2\).
\(E[MSE\mid X]=E[SSE\mid X]/(n-r)=\sigma^2\). Thus residual mean square is unbiased for error variance under the stated assumptions.
This argument identifies the precise role of the denominator. The fitted model uses \(r\) independent directions to estimate its mean structure; only the remaining \(n-r\) directions contribute residual variation. Dividing by \(n\) would systematically underestimate the variance because it would ignore the constraints created by fitting.
Calculating Residual Mean Square
Worked Example: A Simple Linear Regression
A simple linear regression with an intercept is fitted to six observations. Its residuals, in observation order, are \(-1, 2, -1, 1, -2, 1\). Calculate the residual degrees of freedom and \(MSE\). Use the method of counting independent coefficients, and check that the residuals are consistent with the least-squares constraints.
Method. The model has an intercept and one slope, so it estimates \(p=2\) coefficients. The residual degrees of freedom are \(n-p=6-2=4\).
Check the constraints. For an intercept model, the residuals sum to zero: \[ -1+2-1+1-2+1=0. \] For a simple regression with predictor values \(1,2,3,4,5,6\), the residuals are also orthogonal to the predictor: \[ 1(-1)+2(2)+3(-1)+4(1)+5(-2)+6(1)=-1+4-3+4-10+6=0. \] These checks agree with the least-squares residual conditions.
Calculate \(SSE\). Square each residual and add: \[ SSE=(-1)^2+2^2+(-1)^2+1^2+(-2)^2+1^2 =1+4+1+1+4+1=12. \]
Calculate \(MSE\). \[ MSE=\frac{12}{4}=3. \] The residual mean square is \(3\) squared response units. The common mistake avoided here is dividing by six, the sample size, rather than by four, the residual degrees of freedom.
Worked Example: Multiple Regression with Three Predictors
A regression with an intercept and three predictors is fitted to nine observations. The fitted residuals are \(-2,1,0,2,-1,3,-2,1,-2\). Find the residual degrees of freedom, the residual sum of squares, and the residual mean square.
Count estimated coefficients. There are three predictor coefficients and one intercept, so \(p=4\). Therefore, \[ df_{\text{res}}=n-p=9-4=5. \]
Compute the sum of squared residuals. \[ SSE=(-2)^2+1^2+0^2+2^2+(-1)^2+3^2+(-2)^2+1^2+(-2)^2 \] \[ =4+1+0+4+1+9+4+1+4=28. \]
Compute the residual mean square. \[ MSE=\frac{28}{5}=5.6. \] Thus the estimate of error variance is \(5.6\) squared response units. The common mistake avoided is counting only the three predictors and forgetting that estimating the intercept also uses one degree of freedom.
Worked Example: Recovering Residual Mean Square from Data
A simple linear regression with an intercept is fitted to five observations. The predictor values are \(-2,-1,0,1,2\), the fitted values are \(6,8,10,12,14\), and the observed responses are \(7,6,12,10,15\). Calculate \(MSE\), checking that the listed fitted values are consistent with the least-squares residual conditions. Round only the final result to four decimal places.
Find the residuals. Subtract each fitted value from its observed response: \[ e=(7-6,\ 6-8,\ 12-10,\ 10-12,\ 15-14)=(1,-2,2,-2,1). \]
Check the least-squares conditions. The residuals sum to \(1-2+2-2+1=0\). Their product sum with the predictor values is \[ (-2)(1)+(-1)(-2)+0(2)+1(-2)+2(1)=-2+2+0-2+2=0. \] Thus they are orthogonal to both the intercept column and the predictor column, as required for this fitted simple regression.
Calculate \(SSE\) and the residual degrees of freedom. \[ SSE=1^2+(-2)^2+2^2+(-2)^2+1^2=1+4+4+4+1=14, \] and \(df_{\text{res}}=5-2=3\), since the model estimates an intercept and a slope.
Calculate \(MSE\). \[ MSE=\frac{14}{3}=4.6667. \] The final answer is \(4.6667\) squared response units, rounded to four decimal places. The common mistake avoided is dividing by five merely because there are five observations.
Common Traps and Interpretation
The residual mean square estimates the variance of the model errors, not the variance of the responses themselves and not the average squared distance using \(n\) as the divisor. It remains in squared response units. Its square root, sometimes called the residual standard deviation, is in response units, but it is a different quantity from the variance estimate.
A frequent error is to use “number of predictors” as though it always meant “number of estimated coefficients.” For a model with an intercept and \(k\) ordinary numerical predictors, the number of estimated coefficients is \(k+1\), so the residual degrees of freedom are \(n-k-1\). A categorical predictor or interaction can contribute several coefficients; count the independent columns actually estimated, not just the names of the predictors.
Wrong calculation: A model has \(n=18\), an intercept, and four predictor coefficients, with \(SSE=39\). Using \(18-4=14\) gives \(39/14\), because the intercept was omitted from the coefficient count. Correction: There are five estimated coefficients, so \(df_{\text{res}}=18-5=13\), and \(MSE=39/13=3\). The count must include every independent estimated coefficient.
Finally, a small \(MSE\) is not by itself proof that the model is appropriate. The estimate relies on the assumed mean structure and error variance conditions. If the regression assumptions fail, residual mean square may not estimate the intended common error variance well. The calculation is meaningful only in relation to a specified regression model and its assumptions.
Check Your Understanding
For each problem, identify the residual degrees of freedom before calculating the residual mean square.
- A simple linear regression with an intercept is fitted to seven observations, and \(SSE=14\). Find the residual degrees of freedom and \(MSE\). (Answer: \(df_{\text{res}}=5\), \(MSE=2.8\).)
- A model with an intercept and four predictor coefficients is fitted to 12 observations, with \(SSE=42\). Find the residual degrees of freedom and \(MSE\). (Answer: \(df_{\text{res}}=7\), \(MSE=6\).)
- A fitted model has an intercept and two predictor coefficients. Its nine residuals are \(-1,0,2,-2,1,1,-1,0,0\). Compute \(SSE\), residual degrees of freedom, and \(MSE\). (Answer: \(SSE=12\), \(df_{\text{res}}=6\), \(MSE=2\).)
- A design matrix has 15 rows and rank 5, although it has more than five columns. The model's \(SSE\) is 25. What divisor should be used, and what is \(MSE\)? (Answer: divisor \(15-5=10\), \(MSE=2.5\).)
- A full-rank model with an intercept estimates six coefficients from 20 observations, and its errors are independent and normally distributed with common variance \(\sigma^2\). The observed \(SSE\) is 35. Find \(MSE\) and the degrees of freedom of \(SSE/\sigma^2\). (Answer: \(MSE=35/(20-6)=2.5\); \(SSE/\sigma^2\) has a chi-square distribution with 14 degrees of freedom.)