One Residual Plot, Two Different Problems
A residual plot can show more than one pattern at once. A curved band of residuals suggests that the fitted mean relationship is missing structure. A band that widens or narrows suggests that the outcome’s variability changes across predictor values. These features can coexist, and each calls for a different response.
The distinction matters because heteroskedasticity-robust standard errors adjust uncertainty calculations for unequal variance under appropriate conditions; they do not change the fitted mean. If the mean relationship is curved but a straight line is used, robust standard errors do not make that line an adequate description. The diagnostic task is therefore to examine the location and the spread of residuals separately.
For clarity, consider a straight-line mean model \(Y_i=\beta_0+\beta_1x_i+\varepsilon_i\). Here \(Y_i\) is the outcome for observation \(i\), \(x_i\) is its predictor value, and \(\varepsilon_i\) is the error relative to the proposed mean. A useful starting assumption is \(E(\varepsilon_i\mid X_i=x_i)=0\). The variance \(\operatorname{Var}(\varepsilon_i\mid X_i=x_i)\) may be constant or may vary with \(x_i\); the residual plot helps assess that feature.
The fitted residual is \(e_i=y_i-\hat{y}_i\), the observed outcome minus its fitted value. A residual-versus-predictor plot is useful when there is one key predictor; a residual-versus-fitted plot can reveal patterns across the range of predicted outcomes. Neither plot, by itself, proves which model is correct. Look for systematic patterns, not isolated residuals.
Why Mean Curvature Can Look Like Changing Spread
A residual’s magnitude can be large either because the outcome varies widely around its true conditional mean or because the fitted mean is wrong in that region. The distinction can be stated exactly for residuals relative to a fixed proposed mean function \(m_0(x)\). Suppose the true conditional mean is \(m(x)=E(Y\mid X=x)\), and write \(R=Y-m_0(X)\). Then \(R\) is the residual from the proposed mean function before accounting for estimation of its coefficients.
To see why, write \(R=\{Y-m(x)\}+\{m(x)-m_0(x)\}\). Conditional on \(X=x\), the second term is fixed and the first has conditional mean zero. Squaring and taking the conditional expectation makes the cross-product vanish:
Thus, a plot of residual magnitude or squared residuals can reflect both actual outcome variability and mean-model error. In practice, residuals use estimated coefficients, so their summaries are affected by fitting the model as well. The identity still gives the key diagnostic principle: do not infer changing variance from large residuals alone if their average also varies systematically with the predictor.
Worked Example: Separating a Curved Pattern From a Fan
Suppose a hypothetical training program records weekly practice time \(x\), in hours, and a test score \(Y\), in points, for 100 participants. A straight-line model is fitted. The following summaries group its residuals into five equally sized ranges of practice time. The residual standard deviations are descriptive summaries, not formal tests.
| Practice time (hours) | Mean residual (points) | Residual SD (points) |
|---|---|---|
| 0 to less than 2 | +1.0 | 1.0 |
| 2 to less than 4 | −0.5 | 1.5 |
| 4 to less than 6 | −1.0 | 2.0 |
| 6 to less than 8 | −0.5 | 3.0 |
| 8 to 10 | +1.0 | 4.5 |
The mean residuals start positive, become negative in the middle, and return to positive at the high end. That U-shaped sequence suggests that the straight-line fit systematically misses the conditional mean: it tends to predict too low at both ends and too high in the middle. Separately, the residual SD increases from 1.0 to 4.5 points, which suggests that the spread also grows with practice time.
The appropriate diagnosis is both curvature and changing spread, not simply “heteroskedasticity.” The mean residual pattern motivates checking a more flexible mean function. After that change, the residual spread should be examined again to see whether the increasing variability remains.
Targeted Checks for the Mean
A useful mean-model check starts with the pattern in the residual plot and asks what term could plausibly account for it. For a smooth U-shaped pattern against one quantitative predictor, one possible extension is a quadratic term. Centering the predictor at a meaningful value can make the terms easier to interpret:
Here \(c\) is a chosen reference value, such as the midpoint of the observed predictor range. The coefficient \(\gamma\) describes curvature: a nonzero value allows the mean relationship to bend rather than remain straight. A quadratic term is not automatically the right remedy for every pattern. The proposed form should be plausible for the subject matter and consistent with the observed range; a few unusual observations or an abrupt group difference may call for a different investigation.
A direct visual check is to group observations across the predictor range and inspect the average residual in each group. A sustained pattern of averages above and below zero indicates that the fitted mean may be missing structure. A smooth curve added to a residual plot can also help summarize a trend, but it is a diagnostic aid rather than a formal test.
When a specific additional term is proposed, its coefficient can be tested. For example, testing \(H_0:\gamma=0\) asks whether the quadratic term contributes evidence of curvature, conditional on the other terms in the stated model. If the errors may have unequal variance, a heteroskedasticity-robust standard error can be used for this coefficient test, subject to the usual requirements for the chosen inference method. This accounts for uncertainty under unequal variance; it does not establish that the quadratic mean is correct, nor does a nonsignificant result prove that the mean is linear.
Worked Example: Checking a Quadratic Mean Term
In a hypothetical sample of 120 participants, a score in points is modeled using practice time \(x\), in hours. A residual plot from the straight-line fit has a U-shaped average pattern. A quadratic model centered at 5 hours is fitted:
Suppose the estimated coefficient for the squared term is \(\hat{\gamma}=0.080\) points per hour squared, and its robust standard error is \(SE(\hat{\gamma})=0.025\) points per hour squared. For a large-sample robust Wald check of \(H_0:\gamma=0\), calculate:
For a two-sided standard normal reference distribution, this gives \(p\approx0.0014\). The data provide evidence that the quadratic term contributes to the conditional mean, given the specified model and the assumptions supporting this large-sample robust test. The positive estimate indicates upward curvature over the modeled range; it is not, by itself, evidence that every aspect of the mean model is correct.
The robust standard error is used here because the residual plot also suggests unequal spread. It protects this coefficient’s uncertainty calculation against that variance pattern under appropriate conditions. It does not remove the curvature: the mean-model change comes from adding \((x_i-5)^2\), not from using a robust standard error.
Recheck Spread After Revising the Mean
Once a mean-model change is made, repeat the residual checks using the revised fitted values. First inspect whether the residual averages across the predictor range are now reasonably close to zero without a systematic shape. Then inspect whether the residual spread is roughly stable or still changes. These questions should not be collapsed into one judgment.
Residual standard deviations by group are easy to read, but they do not tell the whole story. In particular, groups should have enough observations for their summaries to be informative, and residuals are not independent observations of the underlying errors after fitting. Treat grouped summaries as diagnostic evidence and use a formal variance test only when it addresses the specific question and its assumptions are suitable.
If the curvature pattern largely disappears but the residual spread still widens, the mean may have improved while unequal variance remains. Robust standard errors may then be relevant to inference about coefficients in the revised mean model. They do not make residual spread constant or necessarily improve prediction. If the residual averages still show a pattern, further mean-model work is needed before treating robust inference as a complete analysis.
Worked Example: Curvature Improves, Unequal Spread Remains
Return to the hypothetical practice-time data. After fitting a quadratic mean, suppose the five residual-group means are \(+0.1,-0.2,0.0,+0.2,-0.1\) points. The residual SDs for the same groups are \(1.1,1.6,2.2,3.2,4.6\) points.
The group means are now close to zero, with no evident U-shaped sequence. That is consistent with the quadratic term having addressed the main average pattern. The SDs, however, still increase substantially. The ratio of the largest to smallest squared SD is:
This large descriptive ratio supports a closer look at unequal spread; it is not, on its own, a formal significance test. The two findings should be reported separately: the residual averages no longer show clear curvature, while their spread still appears to increase across practice time. Robust standard errors may be considered for coefficient inference in the revised model, but they do not eliminate the remaining variance pattern.
A Practical Diagnostic Sequence
Look for a curved or otherwise systematic pattern in residuals against the predictor or fitted values. If present, investigate the mean specification rather than labeling the pattern only as unequal variance.
Compare the vertical spread across the predictor range, using plots and, if helpful, grouped standard deviations. A pattern in spread is evidence about variance, not a substitute for checking residual averages.
Fit a plausible added term, such as a quadratic term for a smooth bend, and assess its contribution using an inference method appropriate for the variance conditions.
Check whether the mean pattern has diminished and whether changing spread remains. Choose any subsequent inference adjustment for the model that is actually being used.
Common Mistake: Treating Robust Standard Errors as a Model Repair
A common mistake is to see a curved residual pattern, calculate robust standard errors, and conclude that the model is now adequate. The reasoning is wrong because a robust standard error changes an estimate of uncertainty; it does not change the fitted values, remove bias from an incorrect conditional mean, or make the residual averages pattern-free.
The correction is to diagnose mean and variance separately. Use residual location and a targeted mean-model check to investigate curvature. Then recheck the spread in the revised model and use robust inference when unequal variance is a concern and the required conditions are reasonable. Robust inference cannot compensate for every problem: for example, it does not by itself address dependence between observations or guarantee that a chosen mean function is appropriate.
Check Your Understanding
Use the distinction between residual location and residual spread to answer these questions.
- In a residual plot, the group means follow a U shape while the residual SDs are similar across groups. Which model feature should be investigated first, and why?
- Why can a plot of squared residuals suggest changing spread when the mean function is actually curved?
- A quadratic-term estimate is positive and its robust test gives a small p-value. What does this indicate, and what does it not establish?
- After adding a plausible mean term, residual group means are near zero but group SDs still increase. State the two diagnostic conclusions separately.
- Explain why replacing a conventional standard error with a robust one does not, by itself, repair a curved mean relationship.