Constant Conditional Variance of Errors

Why Conditional Variance Matters

A regression model describes how the response changes with a predictor, but a fitted mean relationship does not tell us how variable responses are around that mean. For example, the average claim-processing time might rise with claim size while the range of processing times also becomes wider. The changing spread is a feature of the model’s errors, not a change in the definition of the regression slope.

The key question is whether the response has the same conditional variance at each predictor value. A constant conditional variance is called homoscedasticity. If the variance changes with the predictor, the errors are heteroscedastic. The comparison is made across conditional distributions, not by asking whether every observed residual has the same magnitude.

Definition: In the simple linear regression model \(Y_i=\beta_0+\beta_1X_i+\varepsilon_i\), suppose \(E[\varepsilon_i\mid X_i=x]=0\) for predictor values \(x\) in the model’s range. The errors are homoscedastic if there is a constant \(\sigma^2\) such that \(\operatorname{Var}(\varepsilon_i\mid X_i=x)=\sigma^2\) for every such \(x\). If this conditional variance depends on \(x\), the errors are heteroscedastic.

Here \(Y_i\) is the response for observation \(i\), \(X_i\) is its predictor, and \(\varepsilon_i\) is the error from the conditional mean \(\beta_0+\beta_1X_i\). The condition \(E[\varepsilon_i\mid X_i=x]=0\) means that the conditional mean of the response is the line at each predictor value. It is important to state this condition: if the conditional mean is not correctly represented by the line, deviations from it can mix mean-model error with variation around the true conditional mean.

The condition concerns variance, not the entire shape of the error distribution. Homoscedastic errors need not be normally distributed, and the conditional distributions need not have identical shapes. They must have the same finite variance. Conversely, a changing variance is heteroscedasticity even if the conditional means still lie exactly on a straight line.

Conditional Variance of the Response

Under the zero conditional mean condition, the response’s conditional variance equals the error’s conditional variance. To see why, fix \(X_i=x\). The quantity \(\beta_0+\beta_1x\) is then a constant, so adding it shifts the response distribution without changing its spread.

\begin{aligned} \operatorname{Var}(Y_i\mid X_i=x) &=\operatorname{Var}(\beta_0+\beta_1x+\varepsilon_i\mid X_i=x)\\ &=\operatorname{Var}(\varepsilon_i\mid X_i=x). \end{aligned}

Thus homoscedasticity can equivalently be described as constant conditional variance of the response across predictor values. A variance is a squared-spread measure; the corresponding conditional standard deviation is its square root. If the variance increases by a factor of four, for example, the standard deviation increases by a factor of two, not four.

Important distinction: Homoscedasticity does not mean that observed responses or residuals are equal, nor that their absolute deviations are identical. It means that the underlying conditional variance is the same at each predictor value. Individual observations can still lie close to or far from their conditional mean.

Worked Examples

Worked Example: Comparing Two Conditional Distributions

A hypothetical model for time to resolve a service request is \(Y=12+2X+\varepsilon\), where \(X\) is the request’s complexity score. At \(X=2\), the error has a discrete distribution: \(-2\) with probability \(0.25\), \(0\) with probability \(0.50\), and \(2\) with probability \(0.25\). At \(X=5\), the error is \(-1\) or \(1\), each with probability \(0.50\). At each predictor value, determine the conditional mean, error variance, and whether the model is homoscedastic. The stated distributions are the complete conditional error distributions; no interest or mortality assumptions apply.

Method: calculate the conditional mean and variance separately at each \(X\). At \(X=2\), the conditional mean of the error is

E[\varepsilon\mid X=2] =(-2)(0.25)+(0)(0.50)+(2)(0.25)=0.

The conditional variance is therefore the expected squared error:

\operatorname{Var}(\varepsilon\mid X=2) =(-2)^2(0.25)+0^2(0.50)+2^2(0.25) =1+0+1=2.

The conditional mean response at \(X=2\) is \(12+2(2)+0=16\). At \(X=5\), the error mean is \((-1)(0.50)+(1)(0.50)=0\), and its variance is

\operatorname{Var}(\varepsilon\mid X=5) =(-1)^2(0.50)+(1)^2(0.50)=1.

The conditional mean response at \(X=5\) is \(12+2(5)+0=22\). The variances, 2 and 1, differ, so the errors are heteroscedastic. The conditional response standard deviations are \(\sqrt{2}\approx1.414\) and \(1\), respectively.

Check: Both conditional error means are zero, and the response variances equal the corresponding error variances because the conditional means are constants at each fixed \(X\). Common mistake avoided: comparing the conditional means, 16 and 22, to decide whether the spread is constant. A changing mean does not itself imply heteroscedasticity; compare variances.

Worked Example: Interpreting a Variance Function

A hypothetical model for the dollar amount of a small equipment repair has conditional mean \(E[Y\mid X=x]=80+5x\) and conditional error variance \(\operatorname{Var}(\varepsilon\mid X=x)=9+4x\), for \(x\) between 0 and 10. Find the conditional mean, variance, and standard deviation at \(x=1\) and \(x=4\), then interpret the change. Treat the stated variance function as the model; no distributional shape is assumed.

Method: substitute each predictor value into the mean and variance functions. At \(x=1\),

E[Y\mid X=1]=80+5(1)=85, \qquad \operatorname{Var}(Y\mid X=1)=9+4(1)=13.

The conditional standard deviation is \(\sqrt{13}\approx3.606\) dollars. At \(x=4\),

E[Y\mid X=4]=80+5(4)=100, \qquad \operatorname{Var}(Y\mid X=4)=9+4(4)=25.

The conditional standard deviation at \(x=4\) is \(\sqrt{25}=5\) dollars. The conditional mean rises by 15 dollars, while the conditional variance rises from 13 to 25 dollars squared and the standard deviation rises from about 3.606 to 5 dollars. Since the conditional variance changes with \(x\), the errors are heteroscedastic and the response spread is greater at \(x=4\).

Check: The standard deviations square back to the calculated variances: \((\sqrt{13})^2=13\) and \(5^2=25\). Common mistake avoided: reporting 25 as the standard deviation at \(x=4\). It is the variance; the standard deviation is its square root.

Worked Example: Comparing Grouped Residual Spreads

An analyst fits a linear mean model to hypothetical repair-duration data and groups the fitted residuals by the observed complexity score. For three low-score observations, the residuals are \(-2,0,2\) hours; for three high-score observations, they are \(-4,0,4\) hours. Compare the sample variances and standard deviations within the two groups. Use the usual sample variance with denominator \(n-1\). The residual lists are the given data; do not assume a particular error distribution.

Method: compute each group’s mean, then its sum of squared deviations divided by \(n-1\). The low-score residual mean is \((-2+0+2)/3=0\). Its sample variance and standard deviation are

s^2_{\text{low}}=\frac{(-2)^2+0^2+2^2}{3-1} =\frac{8}{2}=4, \qquad s_{\text{low}}=\sqrt{4}=2\text{ hours}.

The high-score residual mean is \((-4+0+4)/3=0\). Its sample variance and standard deviation are

s^2_{\text{high}}=\frac{(-4)^2+0^2+4^2}{3-1} =\frac{32}{2}=16, \qquad s_{\text{high}}=\sqrt{16}=4\text{ hours}.

The high-score group’s sample variance is four times the low-score group’s, and its sample standard deviation is twice as large. This pattern is consistent with increasing spread as complexity increases. It is a diagnostic clue, not proof that the underlying conditional variances differ: each group has only three residuals, and fitted residuals are estimates of unobserved errors.

Check: Squaring the standard deviations gives the sample variances, and the residual means are both zero. Common mistake avoided: comparing only the residual means. Both groups have mean zero, but their spreads are substantially different.

What Heteroscedasticity Changes

Heteroscedasticity changes how widely responses are distributed around their conditional means. A variance that increases with \(x\) produces a wider conditional distribution at higher predictor values; a variance that decreases with \(x\) produces a narrower one. The variance could also change non-monotonically. There is no requirement that heteroscedasticity take the form of a steadily widening “funnel.”

A plot of residuals against fitted values or the predictor can help reveal changing spread. A roughly even vertical band is consistent with constant variance; a band that widens, narrows, or changes width in another systematic way can signal heteroscedasticity. Such plots are diagnostic rather than decisive. A few unusually large residuals, small groups, or an incorrect conditional mean can make the pattern hard to judge.

Residuals are not the same as errors: errors are deviations from the true conditional mean, while residuals are calculated using an estimated regression line. Therefore, residual plots provide evidence about the model’s error variance but do not directly reveal the true conditional variances. In particular, a curved pattern in residuals may indicate a problem with the mean relationship rather than a variance change alone.

The zero conditional mean condition and constant conditional variance are separate assumptions. A model can have a correctly specified linear conditional mean but heteroscedastic errors. Conversely, errors can have a constant variance while the conditional mean is misspecified. Checking one condition does not establish the other.

State the conditional mean model.
Identify the response, predictor, and error, and check that the model assumes a zero conditional mean for the error.

Compare conditional variances.
Evaluate or estimate the spread at different predictor values. Equal variances indicate homoscedasticity; a systematic difference indicates heteroscedasticity.

Interpret spread on the correct scale.
Variances are in squared response units; take square roots to describe conditional standard deviations in response units.

Constant variance matters because familiar regression calculations for uncertainty commonly rely on it, along with other model assumptions. If conditional variances differ, formulas that treat all observations as having one common error variance may not give appropriate standard errors or inferential conclusions. This is a reason to inspect the spread, not a reason to conclude automatically that the fitted mean line is wrong. Assessment of independence and other distributional conditions is separate.

Key takeaway: Homoscedasticity means \(\operatorname{Var}(\varepsilon_i\mid X_i=x)=\sigma^2\) across predictor values. With a zero conditional mean, this is also the conditional variance of the response. Heteroscedasticity describes a changing response spread, not unequal conditional means or unequal individual residuals.

Check Your Understanding

For each problem, focus on conditional spread rather than on the conditional mean alone.

A model has \(\operatorname{Var}(\varepsilon\mid X=x)=16\) at every predictor value. Find the conditional standard deviation and classify the variance pattern. (Answer: The conditional standard deviation is 4 response units; the variance is constant, so the errors are homoscedastic.)
A repair-cost model specifies \(\operatorname{Var}(\varepsilon\mid X=x)=2+x/2\). Calculate the conditional variance and standard deviation at \(x=2\) and \(x=8\), and state whether the errors are homoscedastic. (Answer: At \(x=2\), variance \(=3\) and standard deviation \(=\sqrt{3}\approx1.732\); at \(x=8\), variance \(=6\) and standard deviation \(=\sqrt{6}\approx2.449\); the errors are heteroscedastic.)
At \(X=3\), an error equals \(-3,0,3\) with probabilities \(0.20,0.60,0.20\). At \(X=7\), it equals \(-2\) or \(2\), each with probability \(0.50\). Calculate both conditional error variances and classify the errors. (Answer: Both conditional means are zero; the variances are \(3.6\) at \(X=3\) and \(4\) at \(X=7\), so the errors are heteroscedastic.)
Two groups of four fitted residuals are \((-1,0,0,1)\) and \((-3,0,0,3)\). Using denominator \(n-1\), find their sample variances and standard deviations. What does the comparison suggest? (Answer: The means are both zero; the sample variances are \(2/3\) and \(6\), and the standard deviations are \(\sqrt{2/3}\approx0.816\) and \(\sqrt{6}\approx2.449\). The second group has greater residual spread, consistent with but not proving changing conditional variance.)
Two conditional error distributions both have mean zero and variance 9, but one is symmetric and the other is skewed. Are they homoscedastic across those two predictor values? (Answer: Yes, their conditional variances are equal, so they are homoscedastic across those values; equal variance does not require identical distributional shape.)

What You'll Learn