Interaction Tests Versus Within-Subgroup Tests

Two Questions That Can Produce Different Test Results

When a trial reports a treatment effect in a biomarker-defined subgroup, it is tempting to ask whether that effect is statistically significant and then infer that the treatment works differently for people with and without the biomarker. Those are separate questions. A within-subgroup test asks whether the treatment effect in one subgroup differs from zero. An interaction test asks whether treatment effects differ between subgroups. Neither test answers both questions.

This distinction matters in adaptive trials that may enrich recruitment, continue a selected population, or compare several candidate populations. The test must match the claim: evidence of benefit in a subgroup is not, by itself, evidence that the subgroup benefits more than the rest of the population.

For a continuous outcome, a useful working model is

where \(T_i=1\) for assignment to treatment and \(T_i=0\) for control. In this parameterization, the treatment effect for \(B=0\) is \(\tau_0=\tau\), and the treatment effect for \(B=1\) is \(\tau_1=\tau+\delta\). Thus the interaction is \(\delta=\tau_1-\tau_0\). The within-positive-subgroup null is \(H_0:\tau+\delta=0\), whereas the interaction null is \(H_0:\delta=0\). These are not equivalent null hypotheses.

Assumptions and Standard Errors

The calculations below use a continuous endpoint, independent participants, and random assignment to treatment within each biomarker stratum. Assume the endpoint has a common variance \(\sigma^2\) in the four treatment-by-biomarker cells and is normally distributed within cells. For the worked examples, \(\sigma\) is treated as known so that the tests use standard normal statistics. In an actual trial, \(\sigma\) is usually estimated; an appropriate \(t\)-based or model-based test and its degrees of freedom would then be used.

Let \(n_b\) be the total number assigned within biomarker stratum \(b\), with half assigned to each arm. Write \(\hat{\tau}_b=\bar{Y}_{b,T}-\bar{Y}_{b,C}\) for the estimated treatment-minus-control mean difference in that stratum. With \(n_b/2\) participants per arm,

Because the two biomarker strata contain distinct participants, their treatment-contrast estimates are independent under these assumptions. Consequently, the estimated interaction is \(\hat{\delta}=\hat{\tau}_1-\hat{\tau}_0\), with

The two-sided \(z\)-statistics are \(Z_1=\hat{\tau}_1/SE(\hat{\tau}_1)\) for the within-positive-subgroup test and \(Z_{\mathrm{int}}=(\hat{\tau}_1-\hat{\tau}_0)/SE(\hat{\delta})\) for the interaction test. At significance level \(\alpha=0.05\), each test rejects its own null when the absolute statistic exceeds 1.96, provided that test is the prespecified test being used at that level. If the analysis includes multiple populations or data-dependent selection, the error control must also reflect that design.

Worked Examples: The Conclusions Need Not Match

Power Depends on the Question and the Design

Power is the probability that a specified test rejects its null under a specified true effect and design. For a two-sided normal test at level \(\alpha\), let \(\lambda\) be the true effect divided by the standard error. Its power is

where \(\Phi\) is the standard normal cumulative distribution function. For the subgroup test, \(\lambda=\tau_1/SE(\hat{\tau}_1)\). For the interaction test, \(\lambda=(\tau_1-\tau_0)/SE(\hat{\delta})\). The relevant effect size is therefore different for the two tests.

For the first example, the positive-subgroup noncentrality is \(6/2=3\), giving approximate power 0.851 at two-sided \(\alpha=0.05\). The interaction noncentrality is \(4/2.309=1.732\), giving approximate power 0.410. These powers refer to different alternatives: a 6-point positive-subgroup effect and a 4-point difference between subgroup effects. They do not imply a universal ranking of the tests.

When effects are equal across strata, an interaction test has no power beyond its type I error rate for that particular alternative, while a subgroup test can have high power if that common effect is large enough. Conversely, opposing or strongly separated effects can make the interaction signal large even when neither individual subgroup estimate is precise enough to pass its own test. A small or rare biomarker stratum also increases its treatment-contrast standard error, which can weaken both its subgroup test and the interaction test that uses that contrast.

Interpretation, Diagnostics, and Limitations

Before interpreting either test, verify that the treatment contrast and interaction are defined on the intended scale. In the examples, effects are differences in mean outcome points. For binary outcomes, an interaction on the risk-difference scale need not agree with an interaction on the odds-ratio or log-odds scale. A statement that treatment effects “differ” is incomplete unless the effect scale is clear.

Check the treatment and control counts in each stratum, the estimated effects with their standard errors, and whether the direction and magnitude of the estimates agree with the reported test. A very imprecise estimate in a small stratum can produce unstable conclusions. If the model assumes a common variance, substantial variance differences across cells can make the displayed standard errors inappropriate; a variance-robust or otherwise suitable analysis may be needed. These checks help assess whether the specified model and calculation fit the trial data, but they do not convert a non-significant test into evidence of equality.

The key comparison is between the questions, not which test is “better.” Use a within-subgroup test to assess whether the treatment effect in a particular subgroup differs from its null value. Use an interaction test to assess whether the subgroup effects differ from one another. If the goal is to claim benefit in a selected population and also claim that benefit is greater there than elsewhere, the claims require evidence for their respective hypotheses and an analysis plan that controls error rates appropriately.

In an adaptive trial, selecting a subgroup using interim outcomes and then applying an ordinary unadjusted test to that same subgroup can invalidate the nominal \(p\)-value. The interaction test is not automatically protected from selection either. Valid randomization comparisons after enrichment and any prespecified selection or multiplicity procedure must govern the final inference. Finally, these tests describe differences in the chosen endpoint and effect scale; they do not by themselves establish a biological mechanism or prove that the biomarker causes treatment response.