Why Subgroup Claims Need Interaction Evidence

Why Two Subgroup Tests Do Not Compare Subgroups

Suppose a randomized trial suggests that a treatment may help people with lower baseline values more than those with higher baseline values. A common first analysis is to test the treatment effect separately in each group. If one test is statistically significant and the other is not, it can be tempting to conclude that the treatment works differently across groups. That conclusion does not follow from those two tests.

A test within a subgroup asks whether the treatment effect in that subgroup is zero. A claim that effects differ asks whether the effect in one subgroup is different from the effect in another. These are distinct questions and require distinct hypotheses. The direct comparison is the treatment-by-subgroup interaction contrast.

Definition: For a baseline subgroup indicator \(G\), the treatment effect modification contrast is the difference between the treatment effects in the subgroups. It is also called the treatment-by-subgroup interaction contrast. Evidence that this contrast differs from zero is evidence that the estimated treatment effects differ across the specified subgroups.

Model and Hypotheses

Consider a randomized trial with a continuous outcome \(Y\), where higher values are better. Let \(T=1\) denote assignment to the treatment and \(T=0\) assignment to control. Let \(G=0\) and \(G=1\) denote two subgroups defined using a baseline characteristic measured before randomization. A model with a separate mean for each treatment-by-subgroup combination is

E(Y\mid T,G)=\beta_0+\beta_1T+\beta_2G+\beta_3TG.

Here \(\beta_0\) is the mean for the control group with \(G=0\); \(\beta_1\) is the treatment effect for \(G=0\); \(\beta_2\) is the difference between the control means for \(G=1\) and \(G=0\); and \(\beta_3\) is the difference between the treatment effects in the two subgroups. In this model, the treatment effects are

\tau_0=\beta_1,\qquad \tau_1=\beta_1+\beta_3,\qquad \delta=\tau_1-\tau_0=\beta_3.

The interaction test addresses \(H_0:\beta_3=0\) against \(H_a:\beta_3\ne 0\), unless a directional alternative was specified in advance. By contrast, the separate subgroup tests address \(H_0:\tau_0=0\) and \(H_0:\tau_1=0\). Neither separate test directly tests whether \(\tau_0=\tau_1\).

For the usual regression \(t\)-test and confidence interval, assume the model gives the correct mean for each of the four treatment-by-subgroup cells, observations are independent, and the errors have mean zero and a common variance. Normal errors give exact \(t\)-inference; with sufficiently large samples, the usual regression inference is often justified approximately. The subgroup must be defined from baseline information, not from a variable changed by treatment. Random assignment supports comparing assigned treatment groups within each baseline subgroup, but it does not guarantee that an estimated interaction is precise.

Deriving the Direct Contrast

The interaction contrast is a difference of two treatment contrasts. Subtracting the \(G=0\) treatment effect from the \(G=1\) treatment effect cancels the common treatment coefficient:

(\beta_1+\beta_3)-\beta_1=\beta_3.

This algebra explains why the interaction coefficient, rather than a comparison of whether two p-values fall on opposite sides of 0.05, answers the question of whether effects differ. Its uncertainty must also be considered. In general, if estimates \(\hat{\tau}_1\) and \(\hat{\tau}_0\) have covariance \(\operatorname{Cov}(\hat{\tau}_1,\hat{\tau}_0)\), then

SE(\hat{\tau}_1-\hat{\tau}_0) = \sqrt{SE(\hat{\tau}_1)^2+SE(\hat{\tau}_0)^2 -2\operatorname{Cov}(\hat{\tau}_1,\hat{\tau}_0)}.

A regression model calculates this uncertainty using the estimated covariance of its coefficients. In the binary-subgroup model above, the standard error of \(\hat{\beta}_3\) is used for the direct interaction test. A confidence interval for \(\beta_3\) shows which differences in treatment effects are reasonably compatible with the data.

Worked Example: One Subgroup Test Is Significant, but the Interaction Is Not

In a hypothetical trial, higher outcome scores indicate better health. The model estimates a treatment effect of 6.0 points in subgroup \(G=0\), with \(SE=2.0\), and an effect of 2.0 points in subgroup \(G=1\), also with \(SE=2.0\). The estimated interaction contrast, defined as the effect in \(G=1\) minus the effect in \(G=0\), is \(-4.0\) points, with \(SE=2.83\). Suppose the residual degrees of freedom are 196 and the significance level is \(\alpha=0.05\).

Within \(G=0\), the test statistic for no treatment effect is

t=\frac{6.0-0}{2.0}=3.00.

Its two-sided p-value is about 0.003, so this subgroup's estimated treatment effect is statistically significant at the 0.05 level. Within \(G=1\),

t=\frac{2.0-0}{2.0}=1.00,

with a two-sided p-value of about 0.319. This subgroup's result is not statistically significant at the 0.05 level. But these tests compare each effect with zero, not with each other.

The direct interaction test is

t=\frac{-4.0-0}{2.83}\approx-1.41.

With 196 degrees of freedom, the two-sided p-value is about 0.160. Thus, these data do not provide statistically significant evidence at the 0.05 level that the treatment effects differ between the two baseline subgroups. The estimated difference is 4.0 points in favor of a larger effect in \(G=0\), but it is too imprecise to distinguish clearly from zero. This result does not establish that the subgroup effects are equal.

Interpreting Evidence for an Interaction

A significant interaction provides evidence against equal treatment effects on the scale and for the subgroups used in the model. It does not by itself show that the difference is clinically important, explain why it occurs, or establish that the same pattern applies to other populations. The estimated contrast and its confidence interval help convey magnitude as well as statistical uncertainty.

For a two-sided 95% confidence interval, the model-based interval has the form

\hat{\beta}_3\ \pm\ t^* SE(\hat{\beta}_3),

where \(t^*\) is the appropriate critical value for the model's residual degrees of freedom. If the interval includes zero, the data do not rule out equal subgroup effects at the corresponding two-sided 0.05 level. If it excludes zero, the interaction test is statistically significant at that level. Neither result should be interpreted without considering the interval's range and the practical importance of differences within it.

Worked Example: A Direct Test Detects a Difference

In a second hypothetical randomized trial, the estimated treatment effect is 1.0 point in \(G=0\) and 7.0 points in \(G=1\). Each estimate has a standard error of 1.5 points. Suppose the model reports an interaction standard error of 2.12 points and has 196 residual degrees of freedom. The estimated interaction, \(G=1\) minus \(G=0\), is

\hat{\beta}_3=7.0-1.0=6.0\text{ points}.

The direct test statistic is

t=\frac{6.0}{2.12}\approx2.83.

The two-sided p-value is about 0.005, which is below 0.05. Using \(t^*\approx1.972\), the 95% confidence interval is

6.0\pm1.972(2.12) =6.0\pm4.18 =(1.82,\ 10.18)\text{ points}.

The interval excludes zero, consistent with the significant interaction test. In this hypothetical trial, the estimated treatment effect is about 6 points greater in \(G=1\) than in \(G=0\); the interval indicates that differences from about 1.82 to 10.18 points are compatible with the model and data at the 95% confidence level. This supports a difference on the outcome-point scale, but further judgment is needed to decide whether that difference matters clinically.

Worked Example: Calculating an Interaction from Four Cell Means

A third hypothetical trial has 25 participants in each treatment-by-subgroup cell. The outcome is a score in which higher values are better. The cell means are 50 for control and 56 for treatment in subgroup \(G=0\), and 60 for control and 62 for treatment in subgroup \(G=1\). Assume each cell has sample standard deviation 10 points. The model uses four cell means, independent observations, and a common within-cell variance.

The estimated treatment effect is \(56-50=6\) points in \(G=0\), and \(62-60=2\) points in \(G=1\). Therefore, the interaction contrast \(G=1\) minus \(G=0\) is

\hat{\delta}=(62-60)-(56-50)=2-6=-4\text{ points}.

There are 100 observations and four estimated cell means, so the residual degrees of freedom are \(100-4=96\). With equal sample sizes and standard deviations, the pooled within-cell variance estimate is \(MSE=10^2=100\) square points. The interaction contrast uses one mean from each of the four cells, so its estimated variance and standard error are

\widehat{\operatorname{Var}}(\hat{\delta}) =100\left(\frac{1}{25}+\frac{1}{25}+\frac{1}{25}+\frac{1}{25}\right) =16, \qquad SE(\hat{\delta})=\sqrt{16}=4\text{ points}.

The direct test statistic is \(t=(-4-0)/4=-1.00\), with 96 degrees of freedom; its two-sided p-value is about 0.320. A 95% interval, using \(t^*\approx1.985\), is

-4\pm1.985(4)=(-11.94,\ 3.94)\text{ points}.

The interval includes zero, so this trial does not provide statistically significant evidence at the 0.05 level that the effects differ. The negative estimate points toward a smaller effect in \(G=1\), but the interval is wide and also includes differences in the other direction. The result is uncertainty about effect modification, not proof of identical effects.

Common Mistakes and Good Reporting

The key mistake is to treat “significant here” and “not significant there” as proof that the subgroup effects differ. A small p-value in one group and a larger p-value in another can arise even when the estimated effects are similar: standard errors, and therefore precision, may differ between groups. The correct question is whether the estimated difference between effects is sufficiently precise to distinguish it from zero or from other values of practical interest.

Wrong reasoning: “The treatment was significant in \(G=0\) but not in \(G=1\), so it works better in \(G=0\).” Correction: Test and report the treatment-by-subgroup contrast directly.
Wrong reasoning: “The interaction test was not significant, so the treatment effects are the same.” Correction: A nonsignificant result is not proof of equality. Report the estimated contrast and its confidence interval to show the uncertainty.
Wrong reasoning: “A significant interaction proves the treatment is beneficial in one subgroup and harmful in the other.” Correction: The interaction tests whether effects differ, not whether either effect differs from zero. Examine the subgroup-specific estimates and their uncertainty as well.

Subgroups and the interaction analysis should be chosen with care. Looking across many possible cut points or subgroups and highlighting only the most striking result can produce misleading evidence. Report which baseline characteristic defines each subgroup, the treatment-effect estimates within them, the interaction contrast with its standard error or confidence interval, and the relevant p-value. State the outcome scale and units so the size and direction of the estimated difference are clear.

Key takeaway: Separate subgroup tests ask whether each subgroup's treatment effect is zero. A treatment-by-subgroup interaction test asks whether those effects differ. A subgroup claim about differing effects requires direct interaction evidence, interpreted alongside the estimated contrast and its uncertainty.

Check Your Understanding

Try these questions using the distinction between a subgroup effect test and a direct interaction test.

In a two-subgroup trial, the treatment effect is significant in \(G=0\) but not in \(G=1\). What hypothesis does the interaction test address that the two subgroup tests do not?
If the estimated effects are 3 points in \(G=0\) and 5 points in \(G=1\), what is the interaction contrast when it is defined as \(G=1\) minus \(G=0\)?
A 95% confidence interval for the interaction contrast includes zero. What can you conclude, and what can you not conclude, about equality of effects?
Why should a subgroup characteristic used for this comparison be measured at baseline?
What additional information, beyond the interaction p-value, should be reported to convey the size and uncertainty of a subgroup difference?