Estimating the Regression Error Variance

Why Estimate Error Variance?

A fitted regression line describes the estimated conditional mean response at each predictor value. Individual observations generally do not lie exactly on that line. The error variance describes the spread of the model errors around the conditional mean; it is not the variance of the predictor values, nor is it the variance of the observed responses without qualification. Estimating it gives a numerical measure of the scatter left after fitting the line.

In simple linear regression, the estimate is calculated from the residual sum of squares and the residual degrees of freedom. A related quantity, the residual standard error, is often easier to interpret because it is expressed in the same units as the response. Keeping the two quantities distinct matters: one is a variance, and the other is a standard deviation.

Model and assumptions: For observations $Y_i$ at predictor values $x_i$, $i=1,\ldots,n$, the simple linear regression model is $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$. For the usual error-variance estimate, assume the errors are independent, have mean zero, and have common variance $\sigma^2$. The predictor values are treated as fixed, and at least two distinct predictor values are needed to fit an intercept and slope. Normal errors are not needed just to calculate the estimate. The formulas below require $n>2$.

The least-squares fitted value at $x_i$ is $\hat{Y}_i=\hat{\beta}_0+\hat{\beta}_1x_i$. The residual is the observed response minus the fitted response, $e_i=Y_i-\hat{Y}_i$. The residual sum of squares, abbreviated RSS, adds the squared residuals. With an intercept and a slope fitted, the standard residual degrees of freedom are $n-2$.

Definition: In simple linear regression with an intercept and a slope, the estimated error variance is $$ s^2=\frac{\mathrm{RSS}}{n-2} =\frac{\sum_{i=1}^{n}e_i^2}{n-2}. $$ The residual standard error is the square root of that variance estimate: $$ s=\sqrt{s^2} =\sqrt{\frac{\mathrm{RSS}}{n-2}}. $$ Here $s^2$ estimates the error variance $\sigma^2$; $s$ estimates the error standard deviation $\sigma$.

From Residuals to the Two Estimates

The calculation has three distinct steps: form each residual, square and add the residuals to obtain RSS, and divide by $n-2$. Only after estimating the variance do you take a square root to obtain the residual standard error. This order also tracks the units. If responses are measured in response-units, residuals have response-units, RSS and $s^2$ have squared response-units, and $s$ has response-units.

The relationship between the two reported quantities follows directly from their definitions. Dividing RSS by $n-2$ gives the average squared residual on the residual-degrees-of-freedom scale. Taking the square root reverses the squaring of the units and converts the variance estimate to a standard-deviation estimate:

\mathrm{RSS}=\sum_{i=1}^{n}e_i^2 \quad\Longrightarrow\quad s^2=\frac{\mathrm{RSS}}{n-2} \quad\Longrightarrow\quad s=\sqrt{s^2}.

This algebraic relationship is useful as a check: $s$ must be nonnegative, and squaring the reported residual standard error must recover the reported error-variance estimate, subject to any rounding. Do not call $s$ the variance estimate; it is the standard error of the residual scale, measured in the response's units.

Worked Example: Calculate Both Estimates from Residuals

A laboratory records a response in units on a calibrated scale for five specimens. A least-squares line with an intercept and a slope has fitted values $20+3x_i$. For the predictor values $-2,-1,0,1,2$, the observed responses are $15,15,22,21,27$, respectively. Find RSS, the estimated error variance, and the residual standard error. Use the response units as given.

Method. Calculate observed minus fitted for each specimen, square each residual, and use $n-2$ for the variance estimate. This avoids using the number of observations as the divisor.

The fitted responses are $20+3(-2)=14$, $20+3(-1)=17$, $20+3(0)=20$, $20+3(1)=23$, and $20+3(2)=26$. Thus the residuals are $15-14=1$, $15-17=-2$, $22-20=2$, $21-23=-2$, and $27-26=1$.

Squaring and adding gives $1^2+(-2)^2+2^2+(-2)^2+1^2=1+4+4+4+1=14$, so $\mathrm{RSS}=14$. There are $n=5$ observations, and the residual degrees of freedom are $5-2=3$. Therefore, $s^2=14/3=4.6667$ squared response units, rounded to four decimal places.

Taking the square root gives $s=\sqrt{14/3}=2.1602$ response units, rounded to four decimal places. As a check, $2.1602^2$ is approximately $4.6665$, which agrees with $4.6667$ up to rounding.

Common mistake avoided: Dividing by $5$ would give $14/5=2.8$, not the standard error-variance estimate for this fitted simple regression. The required divisor here is $n-2=3$.

Interpreting Units and Scale

The estimated error variance is reported in squared response units. For example, if the response is a cost measured in thousands of dollars, then the residuals and residual standard error are in thousands of dollars, while the variance estimate is in squared thousands of dollars. To communicate a typical residual scale in ordinary response units, report $s$, not $s^2$.

A residual standard error describes the estimated size of the model's residual scatter on the response scale. It is not the standard error of the fitted slope, the standard error of the fitted intercept, or automatically a confidence interval for a future observation. Those are different quantities with different calculations and interpretations. Here the task is specifically to estimate the common error variance and express its square root in response units.

Worked Example: Keep Thousands of Dollars and Dollars Straight

A hypothetical insurer models claim payment in thousands of dollars against a risk score. For $n=6$ policies, the residuals from the fitted line, in thousands of dollars, are $1,-1,0,0,-1,1$. Calculate the estimated error variance in squared thousands of dollars and the residual standard error in both thousands of dollars and dollars.

Method. Square the residuals and add them, then divide by $n-2$. Convert the standard error to dollars only after taking its square root. This avoids treating a squared-unit variance as though it were a payment amount.

The squared residuals are $1,1,0,0,1,1$, so $\mathrm{RSS}=1+1+0+0+1+1=4$ in squared thousands of dollars. The residual degrees of freedom are $6-2=4$, giving $s^2=4/4=1$ squared thousands of dollars.

The residual standard error is $s=\sqrt{1}=1$ thousand dollars. Since one thousand dollars is $\$1{,}000.00$, the residual standard error is $\$1{,}000.00$. The variance estimate is not $\$1.00$ or $\$1{,}000.00$: its unit is squared thousands of dollars. In ordinary dollar-squared units, its value is $1\times(1{,}000)^2=\$1{,}000{,}000$ squared dollars.

Check. Squaring the residual standard error of $1$ thousand dollars gives $1$ squared thousand dollars, matching the variance estimate.

Common mistake avoided: Reporting $s^2=1$ as a typical claim amount confuses a squared-unit quantity with a response-scale quantity. The typical residual scale reported here is $s=1$ thousand dollars.

Using a Regression Summary

Often a problem supplies the sample size and RSS directly, rather than listing every residual. The calculation remains the same. Verify that the stated RSS comes from the fitted simple linear regression with an intercept and a slope, identify $n$, calculate $n-2$, and then report both the variance estimate and its square root if requested.

A numerical value for RSS alone is not enough to determine $s^2$: the sample size is also needed. Conversely, knowing $n$ alone is not enough without the residuals or RSS. Be attentive to what the response units are, especially when a summary table uses a scaled unit such as thousands.

Worked Example: Interpret a Summary for Hypothetical Claim Costs

A hypothetical portfolio analysis fits a simple regression with an intercept and a slope to $14$ claim-cost observations. The response is measured in thousands of dollars, and the regression output reports $\mathrm{RSS}=96$ squared thousands of dollars. Calculate the estimated error variance and residual standard error, then express the residual standard error in dollars to the nearest cent.

Method. Use the reported RSS and sample size to find $s^2$, then take the square root and convert the response scale from thousands of dollars to dollars. This avoids taking the square root before using the correct divisor.

The residual degrees of freedom are $n-2=14-2=12$. Thus $s^2=96/12=8$ squared thousands of dollars.

The residual standard error is $s=\sqrt{8}=2.828427\ldots$ thousand dollars. Multiplying by $1{,}000$ converts this to $2{,}828.427\ldots$ dollars. Rounded to the nearest cent, the residual standard error is $\$2{,}828.43$.

Check. In thousands of dollars, $2.828427\ldots^2=8$, matching the estimated variance. In dollar units, the variance corresponds to $8\times1{,}000{,}000=\$8{,}000{,}000$ squared dollars.

Common mistake avoided: Taking $\sqrt{96}$ would ignore the residual degrees of freedom and produce the wrong scale. The square root is taken after calculating $96/12$.

Common Errors and a Calculation Checklist

Several small choices can change the reported answer or its interpretation. Before finalizing a calculation, check the model being fit, the divisor, the square root, and the units.

Using $n$ instead of $n-2$. For the standard simple regression model with an estimated intercept and slope, calculate $s^2=\mathrm{RSS}/(n-2)$, not $\mathrm{RSS}/n$.
Using $n-1$. The familiar sample-variance divisor $n-1$ is not the standard divisor for this simple regression error-variance estimate.
Calling $s$ a variance. The variance estimate is $s^2$; the residual standard error is $s=\sqrt{s^2}$.
Forgetting the square root. RSS divided by $n-2$ gives squared response units. Take the square root to return to response units.
Mixing response scales. If responses are recorded in thousands, then $s$ is also in thousands. Convert it to dollars by multiplying by $1{,}000$, not by changing the variance's units without squaring the conversion factor.
Applying the formula to a different fitted model without checking. The divisor $n-2$ applies to simple linear regression with an intercept and one slope. A different number of fitted parameters changes the residual degrees of freedom.

Key takeaway: For a fitted simple linear regression with an intercept and a slope, calculate $\mathrm{RSS}=\sum e_i^2$, estimate the error variance by $s^2=\mathrm{RSS}/(n-2)$, and calculate the residual standard error as $s=\sqrt{s^2}$. The variance is in squared response units; the residual standard error is in response units.

Check Your Understanding

For each problem, calculate the requested quantity and keep the response scale and units in view.

A simple regression with an intercept and slope is fitted to $7$ observations, with $\mathrm{RSS}=30$ response-units squared. Find the estimated error variance and residual standard error. (Answer: $s^2=6$ response-units squared; $s=\sqrt{6}\approx2.4495$ response units.)
A fitted simple regression uses $9$ observations and has $\mathrm{RSS}=112$ squared centimeters. What are the error-variance estimate and residual standard error? (Answer: $s^2=16$ squared centimeters; $s=4$ centimeters.)
A regression for a hypothetical repair amount uses $10$ observations, with $\mathrm{RSS}=45$ squared hundreds of dollars. Find $s^2$ and express $s$ in dollars, rounded to the nearest cent. (Answer: $s^2=5.625$ squared hundreds of dollars; $s=\sqrt{5.625}\times100\approx\$237.17$.)
A simple regression with an intercept and slope is fitted to $5$ observations with $\mathrm{RSS}=18$. A candidate reports $18/4=4.5$ as the error-variance estimate. Identify the divisor error and give the correct variance estimate and residual standard error. (Answer: The divisor should be $5-2=3$, not $5-1=4$; $s^2=6$ and $s=\sqrt{6}\approx2.4495$ response units.)
Two fitted simple regressions each use an intercept and a slope. Model A has $n=12$, $\mathrm{RSS}=60$; Model B has $n=7$, $\mathrm{RSS}=30$. Compare their estimated error variances and residual standard errors. (Answer: Model A has $s^2=60/10=6$ and $s=\sqrt{6}$; Model B has $s^2=30/5=6$ and $s=\sqrt{6}$. Both estimates are equal.)

What You'll Learn