Residual Strata for Detecting Missing Interactions

Why Separate the Residual Plot by Group?

An additive regression model can fit an overall relationship between a response and a quantitative predictor while concealing the fact that the relationship differs across groups. A single residual-versus-predictor plot may make that structure hard to see: points from groups with different trends are mixed together. Plotting residuals against the predictor separately for each group can expose the difference.

The goal is diagnostic, not confirmatory. A consistent upward residual trend in one group and a downward trend in another is evidence that the fitted mean structure may be incomplete. It does not by itself establish that an interaction is present, identify its cause, or provide a formal test.

Definition: A residual stratum is the subset of observations belonging to one level of a grouping predictor, considered separately in a residual plot. For a quantitative predictor \(x\) and a binary group indicator \(g\), examine the fitted-model residuals \(e_i=y_i-\hat{y}_i\) against \(x_i\) for \(g=0\), and then do the same for \(g=1\). A residual trend that changes across strata can indicate that the fitted model omits an interaction between \(x\) and group.

The Additive Model and the Interaction Alternative

Let \(Y\) be the response, \(x\) a quantitative predictor, and \(g\) a group indicator coded 0 or 1. The additive model gives the two groups a shared slope for \(x\), while allowing their intercepts to differ:

Y_i=\beta_0+\beta_1x_i+\beta_2g_i+\varepsilon_i.

Under this model, the fitted mean is \(\beta_0+\beta_1x\) in group 0 and \((\beta_0+\beta_2)+\beta_1x\) in group 1. The intercepts can differ, but the slopes are both \(\beta_1\). An interaction model adds the product \(xg\):

Y_i=\beta_0+\beta_1x_i+\beta_2g_i+\beta_3x_ig_i+\varepsilon_i.

Now group 0 has slope \(\beta_1\), while group 1 has slope \(\beta_1+\beta_3\). Thus, \(\beta_3\) is the difference in the groups’ slopes: the change in the mean response per one-unit increase in \(x\) differs by \(\beta_3\) response units between group 1 and group 0. This describes a relationship in the model; it does not imply that changing \(x\) causes a change in \(Y\).

Diagnostic principle: If a fitted additive model is adequate for the conditional mean, its residuals should not show a systematic linear trend with \(x\) in one group that differs from the trend in another. Opposing residual trends—one rising and one falling—are a particularly clear clue that the shared-slope assumption may be unsuitable.

The usual regression setup assumes that the observations are independent and that the conditional mean is represented by the fitted model. Equal variance and normally distributed errors are relevant to some standard inferential procedures, but a residual-strata plot is primarily a visual check of mean structure. Before interpreting it, check that the group coding is clear, the \(x\)-axis has the same scale in every stratum, and the groups have enough overlapping \(x\) values to make a comparison meaningful. A trend comparison beyond the groups’ common range is an extrapolation.

How the Additive Fit Relates to Group-Specific Slopes

The residual trends do not generally equal the full interaction slope. The additive model has to choose one common slope, and the least-squares fit balances the group-specific slopes according to how much \(x\) variation each group provides. The following derivation makes that point precise for a model with group-specific intercepts.

Derivation: For group \(g\), let \(\bar{x}_g\) and \(\bar{y}_g\) be its sample means, and define \(S_{xx,g}=\sum_{i\text{ in }g}(x_i-\bar{x}_g)^2\). Let \(b_g=\sum_{i\text{ in }g}(x_i-\bar{x}_g)(y_i-\bar{y}_g)/S_{xx,g}\) be that group’s least-squares slope, assuming \(S_{xx,g}>0\). In the additive fit, the group intercept can be written as \(\bar{y}_g-b\bar{x}_g\) for any proposed common slope \(b\). Therefore, the residual sum of squares is minimized by minimizing \(\sum_g\sum_{i\text{ in }g}[(y_i-\bar{y}_g)-b(x_i-\bar{x}_g)]^2\). Differentiating with respect to \(b\), setting the derivative equal to zero, and solving gives

\(b=\dfrac{\sum_g\sum_{i\text{ in }g}(x_i-\bar{x}_g)(y_i-\bar{y}_g)}{\sum_g S_{xx,g}}=\dfrac{\sum_g S_{xx,g}b_g}{\sum_g S_{xx,g}}.\)

Thus, the additive slope is a weighted average of the group-specific slopes, with weights \(S_{xx,g}\). Within group \(g\), the slope of the residuals against \(x\) is \(b_g-b\): subtracting the additive fit removes a line with slope \(b\) from that group’s fitted line with slope \(b_g\).

This result prevents a common algebraic mistake. Even when the groups have identical, centered \(x\) values, the interaction column \(xg\) is not generally orthogonal to the additive-model predictor \(x\). With equal \(S_{xx,g}\), the common slope is the average of the two group slopes. If those slopes are \(\beta_1\) and \(\beta_1+\beta_3\), the additive fit uses \(\beta_1+\beta_3/2\); its residual trends are \(-\beta_3/2\) in group 0 and \(+\beta_3/2\) in group 1. The omitted interaction is therefore visible as differing residual trends, but not as the full interaction slope in each stratum.

Worked Example: Opposing Trends in a Balanced Design

Suppose a hypothetical data set contains three observations in each of two groups. The \(x\) values are \(-1,0,1\) in both groups, and the responses are generated by \(y=10+2x+3g+xg\). The observations are:

Group \(g\)	\(x\)	\(y\)
0	-1	8
0	0	10
0	1	12
1	-1	10
1	0	13
1	1	16

Each group has mean \(x=0\). Group 0 has mean response 10 and slope 2; group 1 has mean response 13 and slope 3. For each group, \(S_{xx}=(-1)^2+0^2+1^2=2\), so the additive model’s common slope is

b=\frac{2(2)+2(3)}{2+2}=\frac{10}{4}=2.5.

The group means give the additive fitted model \(\hat{y}=10+2.5x+3g\). Its fitted values are \(7.5,10,12.5\) in group 0 and \(10.5,13,15.5\) in group 1. Subtracting these fitted values from the observed responses gives residuals \(0.5,0,-0.5\) in group 0 and \(-0.5,0,0.5\) in group 1. The group 0 residual trend has slope \(2-2.5=-0.5\); the group 1 residual trend has slope \(3-2.5=0.5\). A plot separated by group would show these opposing patterns, suggesting that the shared-slope model has missed structure.

The residuals also satisfy the least-squares constraints for the additive model: their sum is zero within each group, and their overall cross-product with \(x\) is zero. In fact, the within-group cross-products are \(-1\) for group 0 and \(+1\) for group 1, which cancel. The residual trends within individual strata need not be zero just because the overall residuals are orthogonal to the included predictor \(x\).

Unequal Predictor Spread Changes the Residual Trends

If one group has more spread in \(x\) than another, its slope receives more weight in the additive fit. The result can be quite different from simply averaging the group slopes. The groups’ residual trends will still reflect the difference between each group’s slope and the common fitted slope, but their magnitudes need not be symmetric.

Worked Example: Unequal Spread Across Groups

Consider a hypothetical data set with group 0 observations \((x,y)=(-2,3),(-1,4),(1,6),(2,7)\), and group 1 observations \((-1,4),(1,12)\). Assume independent observational units, and use a linear trend within each group as the diagnostic comparison. In group 0 the responses follow \(y=5+x\), so \(\bar{x}_0=0\), \(\bar{y}_0=5\), and \(S_{xx,0}=4+1+1+4=10\). In group 1 they follow \(y=8+4x\), so \(\bar{x}_1=0\), \(\bar{y}_1=8\), and \(S_{xx,1}=1+1=2\).

The additive slope is the weighted average

b=\frac{10(1)+2(4)}{10+2}=\frac{18}{12}=1.5.

Because both groups have mean \(x=0\), the fitted group intercepts are their group means. The additive fitted model is therefore \(\hat{y}=5+1.5x+3g\). The residuals in group 0, ordered by \(x=-2,-1,1,2\), are \(1,0.5,-0.5,-1\): for example, at \(x=-2\), the fitted value is \(5+1.5(-2)=2\), and the residual is \(3-2=1\). Their trend slope is \(1-1.5=-0.5\).

In group 1, the fitted values at \(x=-1,1\) are \(6.5,9.5\), so the residuals are \(-2.5,2.5\). The residual trend slope is \(4-1.5=2.5\). The plot would show a modest downward trend in group 0 and a steeper upward trend in group 1. This asymmetry is not a contradiction: group 0 has \(S_{xx}=10\), compared with \(S_{xx}=2\) in group 1, so its slope has greater influence on the shared additive slope.

As a check, the residual cross-products with \(x\) are \(-5\) for group 0 and \(+5\) for group 1, summing to zero. The pattern within each stratum remains informative even though the overall least-squares constraint holds.

Parallel Residual Patterns and What They Do Not Show

A useful comparison is a situation in which groups have different mean responses but the same slope. If the additive model represents that structure, residuals should not show a systematic difference in trend across the strata. The groups may still be vertically separated in a plot of the raw responses; the relevant question here is whether their residual relationships with \(x\) differ after accounting for the fitted group intercepts and common slope.

Worked Example: Different Group Means but No Residual Slope Difference

Suppose a hypothetical independent sample has \(x=-1,0,1\) in each group. Group 0 has responses \(17,20,23\), and group 1 has responses \(22,25,28\). These values follow the additive model \(y=20+3x+5g\): group 0 has intercept 20 and slope 3, while group 1 has intercept 25 and the same slope 3.

The additive least-squares fit is \(\hat{y}=20+3x+5g\). For group 0, the fitted values are \(17,20,23\), giving residuals \(0,0,0\). For group 1, the fitted values are \(22,25,28\), also giving residuals \(0,0,0\). There is no residual trend difference in these constructed data, even though the group means differ by 5 response units. The mean difference is represented by the group term \(5g\), not by an interaction.

In real data, residuals would rarely all equal zero. The diagnostic question is whether a consistent difference in trend is visible beyond ordinary scatter, not whether every point lies exactly on the same fitted line. A lack of a visible pattern is not proof that the interaction is exactly zero; a small sample or narrow range of \(x\) can make a real difference difficult to detect.

Interpreting the Plot Carefully

Use the same residual scale and the same horizontal scale for every stratum. Look for a systematic direction across the range of \(x\), rather than allowing a single large residual to determine the interpretation. An isolated point may merit investigation, but it is not by itself evidence of a different slope. Also check for sufficient observations and \(x\)-range within each group: without within-group variation in \(x\), a group’s trend cannot be assessed.

A residual pattern can have explanations other than an omitted interaction. It may reflect an unusual observation, a nonlinear relationship, an omitted predictor, or limitations in the available data. A plot is a prompt to examine the model and the data, not an automatic instruction to add a term. In particular, do not infer a causal effect from a group-specific residual pattern.

Common mistake: “The residuals are orthogonal to the included predictor \(x\), so they cannot have a trend with \(x\) inside either group.” This is incorrect. Least-squares orthogonality applies across the fitted data as a whole; within-group cross-products can be nonzero and cancel across groups. Check each stratum separately, and interpret its residual trend as evidence to investigate rather than as a formal test of an interaction.

Fit and identify the model.
Write down the additive mean structure and identify the quantitative predictor and grouping variable whose interaction may be missing.

Make separate residual plots.
Plot the additive-model residuals against the quantitative predictor within each group, using consistent axes.

Compare the trends.
Look for differing directions or slopes, while checking group sample sizes, predictor ranges, and influential observations.

Describe the evidence with limits.
Report a pattern as a clue that the shared-slope assumption may be inadequate. Do not describe the plot as a formal test or as proof of causation.

Check Your Understanding

Answer each question using the definitions and examples above.

In the model \(Y=\beta_0+\beta_1x+\beta_2g+\varepsilon\), what feature of the mean relationship is constrained to be the same in both groups?
For two groups with slopes 1 and 5 and equal \(S_{xx}\), what common slope does the additive fit use? What are the two residual trend slopes?
Why can residuals have nonzero cross-products with \(x\) within separate groups even when their overall cross-product with \(x\) is zero?
If one group has much larger \(S_{xx}\) than another, how does that affect the common additive slope?
Name two checks that help determine whether a visible difference in residual trends is interpretable.