Residual Patterns Across Predictor Categories

Why Compare Residuals by Category?

A residual-versus-fitted plot or residual-versus-continuous-predictor plot can be useful, but it may combine observations from distinct groups. If one category tends to have positive residuals and another negative residuals, their patterns can cancel in an overall summary. Likewise, one group may have much more residual variability than another even when the overall spread appears moderate.

Comparing residuals across levels of a categorical predictor is a way to look for these group-specific features. For example, a model predicting appointment duration might leave consistently positive residuals for one appointment type and negative residuals for another. This is evidence that the fitted model has a systematic pattern associated with appointment type; it is not, by itself, evidence that appointment type causes the difference.

A positive residual means the observed response is above its fitted value; a negative residual means it is below. A group mean near zero does not establish that the residuals are well behaved: large positive and negative residuals can balance. It is therefore useful to inspect both center and spread, alongside a plot that shows the individual residuals.

Set Up the Comparison

Suppose a fitted regression model is being used to describe a quantitative response \(Y\). Its residuals are \(e_i=y_i-\hat{y}_i\), and each observation belongs to one of the levels of a categorical predictor \(G\). For each level \(k\), record the number of observations \(n_k\) and summarize the residuals belonging to that level. A group mean residual is

A common measure of group spread is the sample standard deviation of the residuals around that group mean:

These quantities are descriptive summaries of the fitted residuals, not estimates that automatically settle whether a model assumption is true. Small groups give imprecise-looking summaries because a few observations can change them substantially. Always show or inspect the individual points, and report group sizes so that a group with only a few observations is not mistaken for strong evidence of a stable pattern.

A useful display places residuals on the vertical axis and category levels on the horizontal axis, using separate points and, if helpful, boxplots. The zero line is a reference for residual centers. A table of \(n_k\), \(\bar{e}_k\), and \(s_{e,k}\) can make group comparisons more explicit. The plot and table answer related but different questions: summaries compress the data, while the points can reveal outliers, clusters, or other features that summaries hide.

Interpret Centers and Spreads Together

A group residual mean clearly away from zero can indicate that the fitted model systematically underpredicts or overpredicts in that group. If \(\bar{e}_k\) is positive, observations in level \(k\) are higher than their fitted values on average; if it is negative, they are lower on average. Compare both the magnitude and the direction across levels, in the response's units.

Residual spread asks a different question: do observations in one group vary around their fitted values more than observations in another? For a model that assumes a common error variance, a visibly larger residual spread in one category can be a warning that the equal-variance assumption may not describe all groups equally well. The diagnostic identifies a pattern to investigate; it does not establish its cause or amount to a formal test.

A crucial detail is whether the categorical predictor is already represented in the fitted model. If it is omitted, differing group residual centers may indicate that the model has not accounted for group-level response differences. If category indicators are included in an ordinary least-squares model with an intercept, group residual means are constrained to be zero. In that case, comparing residual spreads remains informative, while group means cannot be treated as an independent check for offsets.

To see why, write the residual sum of squares as \(SSE=\sum_i e_i^2\). For any one category \(k\), the model has a group-specific fitted level represented by its intercept and indicators. A small change in that group's fitted level changes each fitted value in the group by the same amount. At the least-squares minimum, the derivative of \(SSE\) with respect to this group level is zero:

Therefore \(\sum_{i:G_i=k}e_i=0\), and dividing by \(n_k\) gives \(\bar{e}_k=0\). This argument requires that the fitted model can vary the fitted level for each category and that the least-squares fit includes those observations. It does not imply equal spreads, normal residual distributions, or the absence of other systematic patterns.

How an Overall Summary Can Conceal Group Patterns

The overall residual mean is a size-weighted combination of group residual means:

Consequently, a zero overall mean only says that the signed residuals balance across the full data set. It does not require every group mean to be zero. More generally, a single overall measure of spread combines within-group variation with differences between group centers, so it cannot tell you which category contributes to the pattern.

A Practical Diagnostic Routine

Common Mistakes

• Wrong reasoning: “The overall residual mean is zero, so every category is well fitted.” Correction: group means can cancel. Calculate or plot the residual center within each level.
• Wrong reasoning: “The category residual means are zero, so there is no category pattern.” Correction: when category indicators are in an ordinary least-squares model, those means are zero by construction. Compare spreads and inspect the individual residuals as well.
• Wrong reasoning: “A wider residual spread proves the category causes greater variability.” Correction: a diagnostic plot describes observed model residuals. It does not identify a causal mechanism, and small groups may produce unstable summaries.
• Wrong reasoning: “The residual standard deviations are similar, so the groups are identical.” Correction: similar spreads say nothing by themselves about group centers or other patterns. Read center, spread, sample size, and individual points together.