Diagnosing Nonconstant Error Variance

Why Residual Spread Matters

A regression line describes the conditional mean of a response, but observations do not usually fall exactly on that line. Residual plots help assess whether the scatter around the fitted line is consistent with the model assumptions used for regression inference. One important feature to inspect is whether the vertical spread of residuals stays roughly constant across fitted values or predictor values.

If the residuals fan out as fitted values increase, or become progressively narrower, the error variance may depend on the predictor. This is called nonconstant error variance, or heteroscedasticity. It matters because standard errors and intervals calculated under a constant-variance assumption can be inaccurate, even if the fitted line is a reasonable description of the conditional mean.

Definition: In the simple linear regression model \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\), the errors have constant variance if \(\operatorname{Var}(\varepsilon_i\mid x_i)=\sigma^2\) for every predictor value \(x_i\). If this variance changes with \(x_i\), the errors have nonconstant variance. The residual for observation \(i\) is \(e_i=y_i-\hat{y}_i\), the observed response minus its fitted value.

A residual-versus-fitted plot places \(\hat{y}_i\) on the horizontal axis and \(e_i\) on the vertical axis. A residual-versus-predictor plot instead uses \(x_i\) horizontally. Either can reveal changing spread. The fitted-value plot is useful when the spread is related to the scale of the expected response; the predictor plot makes the relationship to \(x\) direct. These plots diagnose a pattern; they do not by themselves establish its cause.

Reading Patterns of Changing Spread

With constant variance, residuals should have roughly similar vertical scatter throughout the range of fitted values or predictor values. The plot need not look perfectly uniform: residuals are random, and finite samples naturally have irregularities. The concern is a systematic difference in spread, rather than a few residuals that happen to be larger than others.

Funnel or fan shape: the spread grows across the horizontal axis. This is consistent with error variance increasing with the fitted value or predictor.
Reverse funnel: the spread shrinks across the horizontal axis. This is consistent with error variance decreasing.
Changing spread in bands: the scatter differs across ranges or groups, such as a narrow band followed by a wider band. A sharp change can indicate different variability in different parts of the predictor range.
Changing spread without a simple funnel: the scatter may widen and then narrow, or vary by subgroup. The key question remains whether the vertical spread changes systematically.

A residual plot should also be checked for a changing center. A curved sequence of residuals that moves above and below zero is evidence that the fitted mean relationship may be inadequate. A fan shape, by contrast, is a change in the width of the scatter. Both features can occur together, so it is useful to examine the center and spread separately.

Worked Example: Measuring a Funnel Pattern

A hypothetical analyst groups residuals from a fitted model into three ranges of fitted values. The residuals, in response units, are: low fitted values, \(-1,1,-2,2\); middle fitted values, \(-1,0,1,0\); high fitted values, \(-3,3,-4,4\). Calculate the root-mean-square residual in each group and describe the pattern.

Method. Compare a measure of vertical spread in each fitted-value group. For this calculation, use the root-mean-square residual, \(\sqrt{\sum e_i^2/n}\). It measures average squared distance from zero and is easy to compare when the residuals in each group are centered at zero.

For the low fitted-value group, the sum of squared residuals is \(1+1+4+4=10\). There are four residuals, so the root-mean-square residual is \(\sqrt{10/4}=\sqrt{2.5}\approx1.581\).

For the middle group, the sum of squares is \(1+0+1+0=2\), giving \(\sqrt{2/4}=\sqrt{0.5}\approx0.707\).

For the high group, the sum of squares is \(9+9+16+16=50\), giving \(\sqrt{50/4}=\sqrt{12.5}\approx3.536\).

The spread is smallest in the middle and largest at high fitted values. This is not a steadily widening funnel across all three groups, but it is clear evidence that the spread is not uniform. A residual plot would help show the shape more fully; these summaries alone do not identify the cause.

Check. The calculated spreads are about \(1.581\), \(0.707\), and \(3.536\). Their nonconstant pattern agrees with the relative sizes of the residuals in the three groups. The common mistake is to use only the largest residual as a measure of spread; comparing all residuals gives a more representative summary.

Distinguishing Curvature from Changing Spread

Residuals are deviations from the fitted line, so a pattern in their average level across the horizontal axis can indicate that the line misses part of the mean relationship. Curvature is about where the residuals are centered; heteroscedasticity is about how widely they are scattered. Looking only at the largest positive and negative residuals can confuse the two.

Worked Example: A Curved Center with Similar Spread

In a hypothetical residual-versus-predictor plot, residuals are grouped into three predictor ranges. The residuals are: low \(x\), \(-1,0,1\); middle \(x\), \(1,2,3\); high \(x\), \(-3,-2,-1\). Calculate each group’s residual mean and its root-mean-square deviation from that group mean. Decide whether the main signal is curvature or changing spread.

Method. First compare group means to assess how the residual center changes. Then compare within-group spread around each group mean. Centering the spread calculation on the group mean separates changes in location from changes in width.

For low \(x\), the mean residual is \((-1+0+1)/3=0\). The sum of squared deviations from this mean is \(1+0+1=2\), so the root-mean-square deviation is \(\sqrt{2/3}\approx0.816\).

For middle \(x\), the mean residual is \((1+2+3)/3=2\). The deviations from 2 are \(-1,0,1\), so the root-mean-square deviation is again \(\sqrt{(1+0+1)/3}=\sqrt{2/3}\approx0.816\).

For high \(x\), the mean residual is \((-3-2-1)/3=-2\). The deviations from \(-2\) are \(-1,0,1\), so the root-mean-square deviation is again \(\sqrt{2/3}\approx0.816\).

The centers move from 0 to 2 to \(-2\), while the within-group spread is the same in all three ranges. The changing center, not a changing width, is the main signal. In a plot, this sequence would indicate a systematic curved pattern in the residuals and would prompt concern about the fitted mean relationship.

Check. Each group has the same root-mean-square deviation, \(\sqrt{2/3}\), but the group means differ. The common mistake is to treat the large high-\(x\) residuals as proof of greater variance. Here, the high residuals are shifted downward together; their spread around their group center is unchanged.

Why Nonconstant Variance Affects Standard Errors

In the usual simple linear regression model, observations are assumed to have mean \(\beta_0+\beta_1x_i\), independent errors, and a common error variance \(\sigma^2\). The familiar estimated standard error of the slope is \(s/\sqrt{S_{xx}}\), where \(S_{xx}=\sum_i(x_i-\bar{x})^2\) and \(s^2\) is the residual mean square. This standard error depends on the constant-variance assumption.

To see why, condition on the observed predictor values and suppose the errors are independent, have mean zero, and have variances \(\sigma_i^2\), which may differ across observations. The ordinary least-squares slope can be written as the true slope plus a weighted sum of the errors. The weights depend on how far each \(x_i\) is from \(\bar{x}\).

\hat{\beta}_1 = \beta_1+\frac{\sum_i (x_i-\bar{x})\varepsilon_i}{S_{xx}}.

Because the errors are independent, the variance of a weighted sum is the sum of the squared weights times the individual variances. Therefore,

\operatorname{Var}(\hat{\beta}_1\mid x_1,\ldots,x_n) = \frac{\sum_i (x_i-\bar{x})^2\sigma_i^2}{S_{xx}^2}.

Under constant variance, each \(\sigma_i^2=\sigma^2\). The numerator becomes \(\sigma^2\sum_i(x_i-\bar{x})^2=\sigma^2S_{xx}\), so the slope variance reduces to \(\sigma^2/S_{xx}\). That reduction is what supports the familiar formula. When variances differ, the individual terms are weighted differently; simply estimating one common variance and dividing by \(S_{xx}\) need not give the right result.

The same issue affects standard errors for the estimated mean response and prediction intervals. Those calculations use an estimate of error variance. If the variance actually changes across the range, the reported uncertainty can be too large in some regions and too small in others. The direction is not guaranteed by the word “heteroscedasticity” alone: it depends on where larger variances occur relative to the predictor values and on the quantity being estimated.

Worked Example: Comparing Slope Variances Under Unequal Error Variances

Consider a hypothetical regression with fixed predictor values \(-2,-1,0,1,2\). Errors are independent, have mean zero, and are normally distributed with known variances \(1,1,1,4,4\), respectively. Find the true variance and standard error of the least-squares slope. Then compare them with the variance and standard error obtained by incorrectly treating the common variance as the average of the five error variances.

Method. Use the unequal-variance slope formula derived above. The normality assumption is included here to make the interpretation of a normal interval straightforward; it does not make the unequal variances equal.

The mean predictor value is \(\bar{x}=(-2-1+0+1+2)/5=0\), and \(S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\).

The numerator of the true slope variance is \(4(1)+1(1)+0(1)+1(4)+4(4)=4+1+0+4+16=25\). Thus, \(\operatorname{Var}(\hat{\beta}_1\mid x)=25/10^2=0.25\), and the true standard error is \(\sqrt{0.25}=0.500\).

The average of the five error variances is \((1+1+1+4+4)/5=11/5=2.2\). If this average is incorrectly used as a common variance, the calculated slope variance is \(2.2/10=0.22\), giving a standard error of \(\sqrt{0.22}\approx0.469\).

The incorrect calculation understates the slope variance by \(0.25-0.22=0.03\), or \(0.03/0.25=0.12\), which is 12%. It understates the standard error by about \((0.500-0.469)/0.500=0.062\), or 6.2%.

For illustration, if these known variances and normal errors are used to form a 95% normal interval for the slope, the correct margin of error is \(1.96(0.500)=0.980\). The calculation using the incorrect common variance gives \(1.96(0.469)\approx0.919\). Both intervals would be centered at \(\hat{\beta}_1\), but the second is too narrow for this model.

Check. The larger variances occur at positive predictor values, which receive substantial squared weights in the slope variance calculation. The true variance is therefore \(0.25\), not \(0.22\). The common mistake is to average the error variances and assume that this average reproduces the variance of every regression estimate; the weighted formula shows why it need not.

What a Residual Plot Can and Cannot Establish

A visible funnel is evidence against the constant-variance assumption, not a complete diagnosis of the data-generating process. The plot may reflect a variance that changes with the predictor, a variance that changes with the expected response, or differences among subgroups. A curved mean relationship can also make the residual cloud look wider in some places, so examine both its center and its spread.

A practical reading sequence is:

Check the residual center.
Ask whether residuals remain roughly centered around zero across the horizontal range. A systematic curved movement suggests the mean relationship may be misspecified.

Compare the vertical widths.
Look for a consistent increase, decrease, or other systematic change in scatter. Do not infer changing variance from one isolated large residual.

Consider the inference being reported.
If the spread changes, conventional standard errors and intervals based on a common variance may not have their stated accuracy. A good-looking fitted line alone does not resolve this uncertainty issue.

This distinction matters in actuarial work because a regression may be used to quantify a trend, estimate a mean response, or predict a future response. If the variability changes with the predictor, a single error scale can misrepresent the precision of those results. Some analyses use standard errors designed to allow unequal variances, but the choice and validity of a specific adjustment require attention to the data structure and the inferential goal.

Common Mistakes

Calling every residual pattern heteroscedasticity. Curvature changes the residual center; changing variance changes the residual width. Check both.
Assuming a funnel must widen to the right. A narrowing pattern or a change confined to a band can also indicate nonconstant variance.
Believing a reasonable fitted line guarantees valid intervals. The line can describe the conditional mean reasonably well while the common-variance assumption fails.
Claiming the standard error must be too small. The error can go in either direction. It depends on the pattern of variances and the estimate or prediction being considered.
Treating normal errors as a solution. Normality does not imply equal variances. A normal distribution can have a different variance at different predictor values.

Key takeaway: In a residual plot, changing center and changing spread are different signals. A funnel or other systematic change in width raises concern about nonconstant error variance, which can make conventional regression standard errors and intervals unreliable.

Check Your Understanding

For each problem, separate what the residuals do around zero from how widely they are scattered.

A hypothetical residual plot has root-mean-square residuals \(1.2\), \(1.9\), and \(3.1\) across low, middle, and high fitted-value ranges. What pattern does this suggest? (Answer: Increasing spread, consistent with a funnel and possible increasing error variance.)
Three predictor groups have residual means \(0\), \(-1.5\), and \(0\), while their within-group root-mean-square deviations are \(2\), \(2\), and \(2\). What is the main diagnostic signal? (Answer: A changing residual center, not changing spread; investigate possible curvature in the mean relationship.)
For a regression with \(S_{xx}=8\) and independent errors of variances \(1,1,4,4\), suppose the squared centered predictor values \((x_i-\bar{x})^2\) are \(1,1,3,3\), respectively. Calculate the true slope variance. (Answer: \((1+1+12+12)/8^2=26/64=0.40625\).)
A residual plot is centered around zero across the predictor range, but its scatter is much wider at large fitted values. Which conventional inference assumption is in question, and what can be affected? (Answer: Constant error variance is in question; conventional regression standard errors and intervals may be inaccurate.)
Suppose the residuals at high predictor values are mostly negative, but their spread around their group mean matches the spread at low predictor values. Is this primarily evidence of nonconstant variance? (Answer: No; the shifted residual center points toward a mean-pattern problem such as curvature, while the similar within-group spread does not show changing variance.)

What You'll Learn