Constant Error Variance and Coefficient Precision

Why the Spread Around a Regression Line Matters

A regression line describes how the conditional mean of a response changes with a predictor. But observations generally do not fall exactly on that line. The amount of random spread around it matters: it determines how precisely the regression coefficients can be estimated. Two data sets can have the same predictor values and the same fitted line, yet provide very different coefficient precision if their errors have different variances.

This tutorial focuses on the constant-variance assumption in simple linear regression and its role in standard-error calculations. The results are conditional on the observed predictor values. The ideas are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Definition: In the simple linear regression model $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$, the errors have constant conditional variance if $$ \operatorname{Var}(\varepsilon_i\mid x_1,\ldots,x_n)=\sigma^2 $$ for every observation $i$, where the same finite value $\sigma^2$ applies at every predictor value. Since the conditional mean of each error is zero, this is also the condition $\operatorname{Var}(Y_i\mid x_1,\ldots,x_n)=\sigma^2$. It means the spread around the regression mean is constant; it does not mean that the mean response itself is constant.

Assumptions Behind the Usual Precision Formulas

For the standard simple-regression coefficient variance formulas, condition on the observed predictor values $x_1,\ldots,x_n$. Assume the conditional mean model is linear, so $E[\varepsilon_i\mid x_1,\ldots,x_n]=0$; assume errors from different observations are independent; and assume the errors have the same conditional variance $\sigma^2$. Also require $S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2>0$, so the predictor values are not all identical and the regression has rank 2.

Under these conditions, the OLS coefficient estimates have conditional variances

\operatorname{Var}(\hat{\beta}_1\mid x_1,\ldots,x_n)=\frac{\sigma^2}{S_{xx}}, \qquad \operatorname{Var}(\hat{\beta}_0\mid x_1,\ldots,x_n) =\sigma^2\left(\frac{1}{n}+\frac{\bar{x}^{\,2}}{S_{xx}}\right).

Here $\bar{x}=n^{-1}\sum_{i=1}^n x_i$, and $S_{xx}$ measures the spread of the observed predictor values around their mean. The corresponding standard errors, when $\sigma^2$ is known, are the square roots of these variances. In practice $\sigma^2$ is usually unknown. If $n>2$, the regression has rank 2, and the errors are independent with common conditional variance $\sigma^2$, then the usual unbiased estimator is $s^2=\mathrm{RSS}/(n-2)$, where $\mathrm{RSS}=\sum_{i=1}^n e_i^2$ and $e_i=y_i-\hat{y}_i$. Substituting $s^2$ for $\sigma^2$ gives the usual estimated coefficient standard errors.

The condition $n>2$ is needed: when $n=2$ and the predictor values differ, the regression coefficients can be estimated, but $n-2=0$, so this residual-based estimate of $\sigma^2$ is not defined. The variance formulas themselves describe the sampling variance under their stated model assumptions; they do not require errors to be normally distributed. Normality concerns additional small-sample inference, not the constant-variance definition.

Why the Slope Variance Has This Form

The centered-sums expression for the OLS slope is $\hat{\beta}_1=\sum_i(x_i-\bar{x})Y_i/S_{xx}$. Substituting the linear model into this expression separates the fixed slope from the random errors.

Substitute the model.
$\hat{\beta}_1=\sum_i(x_i-\bar{x})(\beta_0+\beta_1x_i+\varepsilon_i)/S_{xx}$. The intercept term vanishes because $\sum_i(x_i-\bar{x})=0$, and $\sum_i(x_i-\bar{x})x_i=S_{xx}$. Thus $\hat{\beta}_1=\beta_1+\sum_i(x_i-\bar{x})\varepsilon_i/S_{xx}$.

Take the conditional variance.
The predictor values are fixed in this calculation. Independence makes the covariances between distinct errors zero, so $\operatorname{Var}(\hat{\beta}_1\mid x)=\sum_i(x_i-\bar{x})^2\operatorname{Var}(\varepsilon_i\mid x)/S_{xx}^2$.

Use the common variance.
When each conditional error variance is $\sigma^2$, the numerator is $\sigma^2\sum_i(x_i-\bar{x})^2=\sigma^2S_{xx}$. Dividing by $S_{xx}^2$ gives $\operatorname{Var}(\hat{\beta}_1\mid x)=\sigma^2/S_{xx}$.

This derivation also shows what changes when the errors have unequal variances. If the errors remain conditionally independent and have zero conditional means, but $\operatorname{Var}(\varepsilon_i\mid x)=\sigma_i^2$ can differ across observations, then

\operatorname{Var}(\hat{\beta}_1\mid x_1,\ldots,x_n) = \frac{\sum_{i=1}^n(x_i-\bar{x})^2\sigma_i^2}{S_{xx}^2}.

The common-variance formula no longer follows: each error variance is weighted by the squared distance of its predictor value from $\bar{x}$. In particular, errors at predictor values far from the mean can have substantial influence on slope precision.

Worked Example: Estimating Coefficient Precision Under Constant Variance

In a hypothetical study of claim-processing time, four offices have workload scores $x=(0,1,2,3)$ and observed mean processing times $y=(2,4,5,7)$, in hours. Fit the OLS line and calculate the usual estimated standard errors. Use the independent, equal-variance regression assumptions.

Method and assumptions. Compute the centered sums, fitted values, and residual sum of squares. The predictor values are not all equal and $n=4>2$, so the regression has rank 2 and the residual variance estimate has positive degrees of freedom. We use the common-variance model to estimate coefficient precision.

First, $\bar{x}=(0+1+2+3)/4=1.5$ and $\bar{y}=(2+4+5+7)/4=4.5$. The centered predictor values are $-1.5,-0.5,0.5,1.5$, so

S_{xx}=2.25+0.25+0.25+2.25=5.

The centered response values are $-2.5,-0.5,0.5,2.5$. Their cross-products with the centered predictor values sum to $3.75+0.25+0.25+3.75=8$. Therefore, $\hat{\beta}_1=8/5=1.6$ and $\hat{\beta}_0=\bar{y}-\hat{\beta}_1\bar{x}=4.5-(1.6)(1.5)=2.1$. The fitted values are $2.1,3.7,5.3,6.9$, giving residuals $-0.1,0.3,-0.3,0.1$. Thus $\mathrm{RSS}=0.01+0.09+0.09+0.01=0.20$.

There are $n-2=2$ residual degrees of freedom, so $s^2=0.20/2=0.10$. The estimated slope standard error is $\sqrt{0.10/5}=\sqrt{0.02}=0.1414$. For the intercept, the variance estimate is $0.10(1/4+1.5^2/5)=0.10(0.25+0.45)=0.07$, so its estimated standard error is $\sqrt{0.07}=0.2646$. Values are rounded to four decimal places.

Check and common mistake. The residuals sum to zero, and their squared values add to the stated RSS of $0.20$. A common error is to divide the RSS by $n$, giving $0.05$, instead of using the two residual degrees of freedom. That would understate the usual variance estimate here. The method avoids that error by identifying the two fitted regression coefficients before calculating the residual variance.

Predictor-Dependent Error Spread

Constant conditional variance is often called homoscedasticity. When the conditional error variance changes with the predictor, the errors are heteroscedastic. A regression model can still have a linear conditional mean when it is heteroscedastic; the issue is that the usual coefficient variance formula and its standard errors no longer correctly describe precision.

For instance, a response may be tightly clustered around its mean at low predictor values and increasingly spread out at high values. In a residual-versus-predictor plot, this can appear as a widening or narrowing band. Such a pattern is a warning rather than a proof: residuals are estimates of unobserved errors, and a small sample may make the pattern hard to distinguish. Also, a changing mean is not itself evidence of changing variance. The question is whether the spread around the conditional mean changes.

Worked Example: Unequal Error Variances and Slope Precision

Suppose four independent hypothetical office-level responses follow the linear conditional-mean model at predictor values $x=(0,1,2,3)$. Their conditional error variances are $(1,1,1,9)$, respectively, and their conditional error means are zero. Calculate the true conditional variance and standard deviation of the OLS slope estimate.

Method and assumptions. Use the unequal-variance formula derived above. The predictor values have $\bar{x}=1.5$, and $S_{xx}=5$, with squared centered values $2.25,0.25,0.25,2.25$. Independence is essential here because it eliminates covariances between errors.

The numerator of the slope variance is

$$ 2.25(1)+0.25(1)+0.25(1)+2.25(9) =2.25+0.25+0.25+20.25=23. $$

The first three contributions sum to $2.75$, and the last contribution is $20.25$. Therefore,

\operatorname{Var}(\hat{\beta}_1\mid x)=\frac{23}{5^2}=\frac{23}{25}=0.92, \qquad \operatorname{SD}(\hat{\beta}_1\mid x)=\sqrt{0.92}=0.9592.

The standard deviation is rounded to four decimal places. If one incorrectly used the homoscedastic formula with $\sigma^2=1$, the result would be $\sqrt{1/5}=0.4472$, substantially smaller than $0.9592$. That substitution is invalid because the fourth observation has conditional error variance 9, not 1. A tempting arithmetic mistake is to report the first three contributions as $3.25$; their correct sum is $2.75$, and the complete numerator is $23$. This explicit check prevents an incorrect variance of $23.5/25$.

Worked Example: Predictor Spread and Slope Precision

A hypothetical analyst can select four workload levels for a controlled study. Compare the slope precision under two designs, $x_A=(0,1,2,3)$ and $x_B=(0,0,3,3)$. Assume independent errors with constant conditional variance $\sigma^2=4$, and the same linear conditional-mean model in both designs.

Method and assumptions. With equal error variances, the conditional slope variance is $4/S_{xx}$. Both designs have four observations and rank 2. For design A, $\bar{x}_A=1.5$, so $S_{xx,A}=2.25+0.25+0.25+2.25=5$. Hence its slope variance is $4/5=0.8$, and its standard deviation is $\sqrt{0.8}=0.8944$.

For design B, $\bar{x}_B=1.5$ as well. The four deviations are $-1.5,-1.5,1.5,1.5$, so $S_{xx,B}=4(2.25)=9$. Its slope variance is $4/9=0.4444$, and its standard deviation is $2/3=0.6667$. Values are rounded to four decimal places. The larger predictor spread in design B gives the more precise slope estimate under these assumptions.

For a further check, the intercept variance factor is $1/n+\bar{x}^2/S_{xx}$. It is $0.25+2.25/5=0.70$ for design A and $0.25+2.25/9=0.50$ for design B. Multiplying by $\sigma^2=4$ gives intercept variances $2.8$ and $2.0$, respectively. This comparison is conditional on the specified model; predictor spread alone does not guarantee better precision if the error variances also change across predictor values.

Check and common mistake. Do not compare only the largest and smallest predictor values; calculate $S_{xx}$ using every observation. The repeated endpoint values in design B give four squared deviations of $2.25$, totaling 9. The method avoids overlooking repeated observations when measuring predictor spread.

What Constant Variance Does—and Does Not—Tell You

The equal-variance condition connects the residual variance estimate to the coefficient standard errors. It does not assert that every residual will have the same magnitude, or that the observed residuals must look perfectly uniform in a finite sample. Random variation can produce uneven residuals even when the model assumptions hold. Conversely, a visually even residual plot cannot prove that the conditional variances are exactly equal.

Keep the assumptions distinct. The linear conditional-mean model concerns $E[Y_i\mid x_i]$. The zero-mean error condition centers the response around that mean. Independence controls cross-observation covariance. Constant conditional variance specifies how the spread compares across predictor values. If any of these conditions fails, identify which variance formula is affected rather than treating all assumptions as interchangeable.

Key takeaway: Under a linear conditional mean, independent errors, and a common conditional error variance, $\operatorname{Var}(\hat{\beta}_1\mid x)=\sigma^2/S_{xx}$. With predictor-dependent error variances, the slope variance instead weights each conditional variance by $(x_i-\bar{x})^2$. The usual homoscedastic standard error should not be used as though that difference were irrelevant.

Check Your Understanding

For each problem, condition on the stated predictor values and use the assumptions given.

In a simple regression with $x=(1,2,3)$, errors are independent with common conditional variance 9. Find $S_{xx}$, the conditional variance of $\hat{\beta}_1$, and its standard deviation. (Answer: $S_{xx}=2$, variance $=4.5$, standard deviation $\approx2.1213$.)
A regression has predictor values $x=(0,1,2)$, independent errors with common conditional variance 4, and a linear conditional mean. Find the conditional variance of the slope estimate. (Answer: $S_{xx}=2$, so $\operatorname{Var}(\hat{\beta}_1\mid x)=2$.)
Four predictor values are $x=(1,1,4,4)$. Errors are independent with common conditional variance 1.44. Find the slope standard deviation. (Answer: $\bar{x}=2.5$, $S_{xx}=9$, variance $=1.44/9=0.16$, standard deviation $=0.4$.)
Let $x=(0,1,2)$, with independent, zero-mean errors having conditional variances $(1,4,1)$. Find the conditional slope variance and standard deviation. (Answer: $S_{xx}=2$; the weighted numerator is $1(1)+0(4)+1(1)=2$, so variance $=2/4=0.5$ and standard deviation $\approx0.7071$.)
A simple regression has two distinct predictor values and $n=2$. Are the coefficients estimable, and is $\mathrm{RSS}/(n-2)$ a defined residual variance estimate? (Answer: The rank-2 coefficients are estimable, but the residual variance estimate is not defined because $n-2=0$.)

What You'll Learn