Common Patterns of Heteroskedasticity

Why Unequal Spread Has More Than One Shape

A regression model describes how the average response changes with predictors, but it also makes an assumption about how responses vary around that average. The variation need not look the same for every fitted value or every kind of observation. For example, household incomes may be tightly clustered among households with low predicted income and much more dispersed among those with high predicted income. Treatment groups may have different residual spreads, or variation in an outcome may depend especially on one predictor such as the size of a business.

These situations share a central feature—conditional variance is not constant—but their diagnostic appearances differ. Recognizing the form of the pattern helps describe the evidence precisely. A pattern alone does not establish its cause, and it does not by itself determine what analysis should follow.

Definition and Model Context

Let \(Y_i\) be a response for observation \(i\), let \(\mathbf{x}_i\) denote its predictor values, and let \(\epsilon_i\) be the error around the conditional mean. The usual linear regression model for the conditional mean is \(Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}+\epsilon_i\). In interpreting residual spread, suppose the conditional mean is adequately represented by this model and observations are independent. The constant-variance assumption says that the conditional error variance is the same at every predictor combination.

Definition: The errors are homoskedastic when \(\operatorname{Var}(\epsilon_i\mid\mathbf{x}_i)=\sigma^2\) for every observation, for one common variance \(\sigma^2\). They are heteroskedastic when \(\operatorname{Var}(\epsilon_i\mid\mathbf{x}_i)\) changes with the predictor values or with another modeled feature such as group membership.

Heteroskedasticity concerns the conditional spread, not the conditional mean. A residual pattern that bends above and below zero may indicate that the mean relationship is misspecified; a residual pattern whose vertical spread changes while remaining centered around zero is evidence more directly associated with unequal variance. These features can occur together, so a changing spread should not be interpreted before considering whether the mean pattern is reasonably represented.

Errors are not observed. A fitted regression produces residuals \(e_i=y_i-\hat{y}_i\), which are observable estimates of the errors and are used to inspect spread. Residuals are constrained by the fitting process and need not reproduce the errors exactly. Therefore, variation in residual groups is descriptive diagnostic evidence, not a direct measurement of the true conditional error variance.

Pattern 1: A Fan Across Fitted Values

A common pattern is a band of residuals that widens or narrows as fitted values increase. A widening fan means that observations with larger predicted means tend to have greater residual variability; a narrowing fan means the opposite. This can occur in settings where larger outcomes naturally have more room for absolute variation, such as income or expenditure. The pattern is about the width of the residual cloud, not a systematic upward or downward drift in its center.

Worked Example: Income Spread Increasing With Predicted Income

Suppose a hypothetical income model predicts annual household income. For illustration, a residual display is summarized by two groups of fitted values. In a lower-fitted-income group, three residuals are \(-1, 0, 1\) thousand dollars. In a higher-fitted-income group, they are \(-3, 0, 3\) thousand dollars. Both groups are centered at zero, but the second group is more spread out.

For the lower-fitted group, the residual mean is \(\bar e=0\). Its sample variance and standard deviation are

s_{\text{low}}^2 =\frac{(-1-0)^2+(0-0)^2+(1-0)^2}{3-1} =\frac{2}{2}=1, \qquad s_{\text{low}}=\sqrt{1}=1

The variance is \(1\) squared thousand dollars, and the standard deviation is \(1\) thousand dollars. For the higher-fitted group,

s_{\text{high}}^2 =\frac{(-3-0)^2+(0-0)^2+(3-0)^2}{3-1} =\frac{18}{2}=9, \qquad s_{\text{high}}=\sqrt{9}=3

The sample standard deviation of residuals is three times as large in the higher-fitted group, and its sample variance is nine times as large. This is a clear illustrative fan-shaped pattern. With only three residuals in each group, it would not establish the population variances or prove that fitted income causes the spread to increase. It suggests that conditional variability may grow at higher fitted incomes and merits careful diagnostic consideration.

A fan can be roughly symmetric around zero, or the spread can grow mainly on one side. A changing spread is still relevant even when the shape is not a neat triangle. The key question is whether the typical magnitude of deviations from the fitted mean changes across the horizontal range.

Pattern 2: Different Spread Across Groups

Heteroskedasticity may occur as a groupwise pattern. For example, residual variability could differ between treatment and control groups, between regions, or between categories of equipment. In such cases, a single residual plot can show bands with different widths, or separate group summaries can reveal different residual dispersions. The group means may also differ; that is a mean contrast, distinct from the question of whether within-group spread differs.

Worked Example: Comparing Residual Spread by Treatment Group

Consider hypothetical residuals, measured in outcome points, from a model that includes treatment-group membership in its conditional mean. The four control-group residuals are \(-1,0,0,1\); the four treatment-group residuals are \(-3,-1,1,3\). Each set has mean zero. Calculate the sample variance using \(s^2=\sum(e_i-\bar e)^2/(n-1)\).

For the control group, \(n=4\), \(\bar e=0\), and the sum of squared deviations is \(1+0+0+1=2\). Thus,

s_{\text{control}}^2=\frac{2}{4-1}=\frac{2}{3}, \qquad s_{\text{control}}=\sqrt{\frac{2}{3}}\approx0.816\text{ points}.

For the treatment group, the squared deviations sum to \(9+1+1+9=20\), so

s_{\text{treatment}}^2=\frac{20}{4-1}=\frac{20}{3}\approx6.667, \qquad s_{\text{treatment}}=\sqrt{\frac{20}{3}}\approx2.582\text{ points}.

The treatment-group sample variance is \((20/3)/(2/3)=10\) times the control-group sample variance, and its sample standard deviation is about \(2.582/0.816\approx3.16\) times as large. These hypothetical summaries illustrate different residual spreads across the two groups. They do not show that treatment caused greater variability: group composition, other predictors, or chance variation could contribute. The comparison is most informative when the conditional mean has been modeled adequately and the groups are otherwise interpreted in context.

Groupwise differences need not mean that all observations in one group have larger residuals than all observations in another. Distributions overlap, and a few extreme residuals can strongly affect a small group’s sample variance. The pattern to look for is a systematic difference in the overall vertical spread, rather than a comparison of single observations.

Pattern 3: Spread Tied to a Particular Predictor

Sometimes variability changes with one predictor in a way that is not adequately summarized by saying only that the spread increases with fitted values. For example, business size might be especially related to variation in sales outcomes even after the model accounts for its average relationship with sales. A plot of residuals against that predictor can reveal a pattern such as lower spread at moderate values and larger spread at both extremes.

This kind of predictor-specific pattern requires care in a multiple-predictor model. Predictors can be associated with fitted values, so a residual-versus-predictor plot may resemble a residual-versus-fitted plot. The diagnostic question is whether the residual spread varies systematically with the particular predictor, conditional on the mean model and the other relevant features—not whether that predictor is the sole cause.

Worked Example: A U-Shaped Spread Across Business Size

Suppose a hypothetical model predicts monthly sales using business size and other predictors. Residuals, measured in thousands of dollars, are grouped into small, medium, and large business sizes. The residuals are: small, \(-2,-1,1,2\); medium, \(-1,0,0,1\); large, \(-4,-2,2,4\). Each group has mean zero. Use the sample variance in each group to compare spread.

For small businesses, the squared deviations sum to \(4+1+1+4=10\); with \(n=4\),

s_{\text{small}}^2=\frac{10}{3}\approx3.333 \quad\text{and}\quad s_{\text{small}}=\sqrt{\frac{10}{3}}\approx1.826\text{ thousand dollars}.

For medium businesses, the squared deviations sum to \(1+0+0+1=2\), giving

s_{\text{medium}}^2=\frac{2}{3}\approx0.667 \quad\text{and}\quad s_{\text{medium}}=\sqrt{\frac{2}{3}}\approx0.816\text{ thousand dollars}.

For large businesses, the squared deviations sum to \(16+4+4+16=40\), so

s_{\text{large}}^2=\frac{40}{3}\approx13.333 \quad\text{and}\quad s_{\text{large}}=\sqrt{\frac{40}{3}}\approx3.651\text{ thousand dollars}.

The sample residual spread is largest for the small and large groups and smallest for the medium group. In a residual-versus-size display, this would look like a U-shaped change in vertical spread, not necessarily a curved change in the residual center. The pattern is compatible with predictor-specific heteroskedasticity, but the few residuals per group provide only a rough description. It would be important to check that the conditional mean is not itself systematically misspecified across business sizes.

A Short Justification: Why Spread Shapes Track Variance

A useful way to understand these patterns is to express an error as a scale that depends on predictors multiplied by a standardized random component. Suppose \(\epsilon_i=\sigma(\mathbf{x}_i)Z_i\), where \(E(Z_i\mid\mathbf{x}_i)=0\) and \(\operatorname{Var}(Z_i\mid\mathbf{x}_i)=1\). Then the conditional mean of the error remains zero, while

E(\epsilon_i\mid\mathbf{x}_i) =\sigma(\mathbf{x}_i)E(Z_i\mid\mathbf{x}_i)=0, \qquad \operatorname{Var}(\epsilon_i\mid\mathbf{x}_i) =\sigma^2(\mathbf{x}_i)\operatorname{Var}(Z_i\mid\mathbf{x}_i) =\sigma^2(\mathbf{x}_i).

Thus a changing scale \(\sigma(\mathbf{x}_i)\) changes variance without requiring the conditional mean to move away from the regression line. If the scale grows with fitted values, the residual cloud can fan out. If it takes different levels by group, group bands can have different widths. If it varies with a specific predictor, the spread can change across that predictor’s values. Residuals are only estimates of errors, but their patterns can provide visible evidence of these changing scales.

Reading Patterns Without Overclaiming

A disciplined description separates what is visible from what remains uncertain. State which axis, group, or predictor is associated with changing spread; say whether the spread grows, shrinks, differs by group, or changes non-monotonically; and note whether the residual center also shows a pattern. Avoid claims about causes unless the study design and analysis support them.

Increasing spread: residuals become more dispersed as fitted values or a predictor increase.
Groupwise spread: one category has visibly wider residual dispersion than another.
Predictor-specific spread: dispersion changes with a particular predictor, perhaps in a curved or non-monotone way.
Uncertain evidence: small groups, outliers, or a misspecified conditional mean can make apparent variance patterns difficult to interpret.

A common mistake is to see a wide residual range at one end of a plot and conclude that the error variance is certainly larger there. The correction is to compare the overall spread across a meaningful range, account for the number of observations, and check that the residuals remain reasonably centered. A single extreme residual can create a wide range without representing a broadly larger variance. Likewise, a curved residual center is not itself proof of heteroskedasticity; it may indicate that the modeled average relationship needs attention.

Key takeaway: Heteroskedasticity means that conditional error variance changes. It may appear as a fan across fitted values, different residual widths across groups, or a predictor-specific pattern. Describe the observed spread precisely, while treating residual evidence as diagnostic rather than conclusive.

Check Your Understanding

Answer each question by distinguishing the residual center from its spread.

A residual cloud remains centered around zero but becomes wider as predicted income increases. What pattern does this suggest?
Two treatment groups have similar residual centers but visibly different vertical spreads. What feature is being compared?
Residual spread is greatest for both the smallest and largest values of a predictor and lowest in the middle. Describe the pattern.
Why are residuals useful for diagnosing variance even though they are not the unobserved errors?
A residual plot bends upward while its vertical spread appears roughly constant. Why should this not automatically be described as heteroskedasticity?