Why Look at Residuals by Group?
A regression model can include two predictors without allowing their relationship with the outcome to depend on one another. For example, an additive model with a continuous predictor and a two-group predictor gives the groups different average levels but requires them to have the same slope. If the true slopes differ, the model may leave a systematic pattern in the residuals. Looking at residuals separately for each group can make that pattern visible.
This is a diagnostic, not a standalone test for an interaction. A visible pattern suggests that the model's mean structure may be incomplete; it does not establish why the pattern occurred, prove that an interaction is needed, or show that a particular change will improve the analysis. The values of the predictors, the fitted model, and the amount of data in each group all matter.
Additive Models and Interactions
Consider a continuous predictor \(x\) and a group indicator \(G\), where \(G=0\) for group A and \(G=1\) for group B. An additive linear model is
Here, the conditional mean is the part before the error term. For group A it is \(\beta_0+\beta_1x\); for group B it is \((\beta_0+\beta_2)+\beta_1x\). The groups can have different intercepts, but both have the same slope, \(\beta_1\). A model with an interaction adds a product term:
In this model the slope for group A is \(\beta_1\), and the slope for group B is \(\beta_1+\beta_3\). Thus, \(\beta_3\) allows the slope to depend on group. More generally, an interaction means that the relationship between one predictor and the outcome varies with another predictor; the variation may concern a slope, a curve's shape, or both.
For these plots to be useful, the fitted model should be stated first. A standard starting point is a regression model with a specified conditional mean and errors with mean zero given the predictors. For ordinary least-squares regression diagnostics, it is also usual to consider whether observations are independent and whether the error variance is reasonably constant across predictor values. Those conditions do not guarantee that the mean form is correct: a residual plot is used in part to look for evidence that it is not. Normal errors are not required simply to inspect the plotted residual pattern.
How to Read a Stratified Residual Plot
Plot residuals against the continuous predictor and distinguish the groups with separate panels, colors, or symbols. A horizontal reference line at zero helps show whether the residuals tend to be positive or negative in different parts of the predictor range. Look for a pattern that is systematic within a group, and compare its direction or shape across groups.
- If the residual trend rises in one group and falls in another, the groups may have different slopes.
- If a curve appears in one group but not another, or their curves bend differently, the groups may have different shapes.
- If a similar curve appears in every group, a shared nonlinear relationship may be a more direct explanation than a group interaction.
- If residuals show no clear pattern, that is compatible with the fitted mean form, but does not prove it is correct.
A plot may suggest a missing interaction even when the overall residual plot looks unremarkable. Opposing group patterns can partly cancel when the groups are combined. Conversely, an apparent difference may be driven by a few observations, unequal predictor ranges, or sparse data. Compare only regions where the groups have data support, and avoid treating a visual impression as a precise estimate.
Worked Example: Opposing Residual Trends Suggest Different Slopes
Suppose a hypothetical analyst fits an additive model to an outcome measured in points, using a predictor \(x\) measured in hours and group A or B. The fitted values follow \(\hat{y}=10+3x\) in group A and \(\hat{y}=12+3x\) in group B. The equal coefficient of \(x\) imposes a common slope. The observed data and residuals are:
| Group | \(x\) (hours) | \(y\) (points) | \(\hat{y}\) (points) | \(e=y-\hat{y}\) (points) |
|---|---|---|---|---|
| A | 0 | 8 | 10 | -2 |
| A | 1 | 13 | 13 | 0 |
| A | 2 | 18 | 16 | 2 |
| B | 0 | 14 | 12 | 2 |
| B | 1 | 15 | 15 | 0 |
| B | 2 | 16 | 18 | -2 |
For example, the first residual is \(8-10=-2\) points, and the last is \(16-18=-2\) points. In group A the residuals rise from \(-2\) to \(0\) to \(2\) as \(x\) increases; in group B they fall from \(2\) to \(0\) to \(-2\). If plotted separately, these opposing trends suggest that the group-specific slopes may differ, which the additive model does not allow.
The arithmetic is consistent with the stated fitted additive model: each group's residuals sum to zero, and the predictor-weighted residual totals cancel across groups, since \((0)(-2)+(1)(0)+(2)(2)=4\) for A and \((0)(2)+(1)(0)+(2)(-2)=-4\) for B. These residual constraints are compatible with least squares for this fitted model. The plot supports investigating a slope interaction; it does not by itself establish that the interaction is important beyond these hypothetical observations.
Why Group-Specific Shapes Matter
Not every group-dependent residual pattern is a difference in straight-line slopes. If residuals bend across \(x\), the mean relationship may be curved. If that bend differs between groups, the model may need a group-dependent shape. The key diagnostic question is whether the pattern changes across groups, not simply whether any pattern exists.
Worked Example: Group-Specific Curvature
Consider hypothetical fitted values from the additive model \(\hat{y}=10+2x+2G\), with \(x\) measured in hours and \(y\) in points. The observations below produce the listed residuals:
| Group | \(x\) (hours) | \(\hat{y}\) (points) | \(e\) (points) | \(y=\hat{y}+e\) (points) |
|---|---|---|---|---|
| A | 0 | 10 | -1 | 9 |
| A | 1 | 12 | 2 | 14 |
| A | 2 | 14 | -1 | 13 |
| B | 0 | 12 | 1 | 13 |
| B | 1 | 14 | -2 | 12 |
| B | 2 | 16 | 1 | 17 |
The entries satisfy \(y=\hat{y}+e\); for instance, group A at \(x=1\) has \(14=12+2\). In group A, the residuals \(-1,2,-1\) form an upside-down U-shaped pattern: the center is higher than both endpoints. In group B, \(1,-2,1\) form a U-shaped pattern: the center is lower than both endpoints. The opposing shapes suggest that a shared additive form may not describe both groups adequately.
For each group, the residuals sum to zero and their predictor-weighted sum is also zero. For A, \((-1)+2+(-1)=0\) and \((0)(-1)+(1)(2)+(2)(-1)=0\); for B, \(1+(-2)+1=0\) and \((0)(1)+(1)(-2)+(2)(1)=0\). Thus these residuals are consistent with the least-squares constraints for group intercepts and a common linear slope. The opposite curvature is a reason to examine a group-dependent shape, not evidence that a specific more complex model must be selected.
Worked Example: The Same Curvature in Both Groups
Now suppose the fitted values are again \(\hat{y}=10+2x+2G\), but both groups have the following residuals at \(x=0,1,2\): \(-1,2,-1\). The corresponding observed outcomes are shown below.
| Group | \(x\) (hours) | \(\hat{y}\) (points) | \(e\) (points) | \(y=\hat{y}+e\) (points) |
|---|---|---|---|---|
| A | 0 | 10 | -1 | 9 |
| A | 1 | 12 | 2 | 14 |
| A | 2 | 14 | -1 | 13 |
| B | 0 | 12 | -1 | 11 |
| B | 1 | 14 | 2 | 16 |
| B | 2 | 16 | -1 | 15 |
In both groups, the same upside-down U-shaped pattern occurs: the center residual is \(2\), greater than the \(-1\) residuals at the endpoints. The residual sum and predictor-weighted sum are zero within each group, so this pattern is also compatible with the least-squares fit of the stated additive model. Because the pattern is shared rather than group-dependent, it points first to a possible common curvature in the relationship with \(x\), rather than to evidence that the groups need different shapes.
A Short Justification: Why Opposing Trends Can Disappear
Suppose residuals show an approximately linear trend within each group. A simple way to describe that trend is the within-group least-squares slope
where \(\bar{x}_g\) and \(\bar{e}_g\) are the group means of the predictor and residual. If \(b_A\) and \(b_B\) differ noticeably, a single pooled trend can conceal the difference. For equal group sizes with comparable predictor spreads, opposite slopes can cancel when pooled.
For the previous slope example, each group has mean \(x=1\) and mean residual \(0\). In A, the numerator is
and the denominator is \((0-1)^2+(1-1)^2+(2-1)^2=2\), so \(b_A=4/2=2\). For B, the numerator is \((0-1)(2)+(1-1)(0)+(2-1)(-2)=-2-2=-4\), with the same denominator \(2\), so \(b_B=-4/2=-2\). The opposing within-group residual slopes are therefore \(2\) and \(-2\) points per hour, even though their pooled contribution cancels. This calculation explains why separate panels can reveal information that a combined plot obscures; it is a diagnostic summary, not an interaction test.
Practical Checks and Common Mistakes
Before interpreting a pattern, check that the groups have enough observations across a useful range of \(x\). If one group occurs mostly at low \(x\) and another mostly at high \(x\), apparent differences may be difficult to distinguish from differences in data coverage. Note whether a pattern is supported by many observations or is largely determined by one extreme point. Also remember that residuals are calculated from the fitted model: they are diagnostic quantities, not the unobserved errors themselves.
- Mistake: Calling every curve an interaction. A similar curved pattern in all groups suggests a shared departure from linearity, not necessarily a group-dependent relationship. Compare shapes across groups.
- Mistake: Reversing a U-shape and an upside-down U. A center point higher than both endpoints is upside-down U-shaped; a center point lower than both endpoints is U-shaped. Check the actual residual values before describing the plot.
- Mistake: Treating a plot as proof. A pattern can be due to sparse data, a few influential observations, or another feature of the data. Describe it as evidence to investigate, not as a conclusive finding.
- Mistake: Assuming an additive model permits different slopes. A group main effect changes the fitted group level; it does not make the slope vary. A slope that depends on group requires a product term in the mean model.
Check Your Understanding
Use the definitions and examples above to answer the following questions.
- In a model with \(G=0\) for group A and \(G=1\) for group B, what do the coefficients of \(x\) and \(xG\) represent in a model that includes both terms?
- Residuals in one group rise across \(x\), while residuals in the other group fall. What feature of the additive model might this pattern suggest is missing?
- Residuals at three increasing predictor values are \( -2, 1, -2 \). Is the center above or below the endpoints, and how should the shape be described?
- Both groups show residuals \(-1,2,-1\) across three increasing values of \(x\). What does this shared pattern suggest, and why is it not by itself evidence of an interaction?
- Give one reason why a visual difference between groups might be unreliable even if their plotted residual trends appear different.