Treatment-Effect Heterogeneity and Subgroup-Specific Precision

Why Overall Precision Does Not Guarantee Subgroup Precision

A trial can estimate an average treatment effect precisely while providing little information about whether that effect differs across patient groups. The distinction matters in model-based endpoint planning: a sample size adequate for the primary overall contrast may be inadequate for a treatment-by-subgroup interaction. This is especially likely when the subgroup of interest is uncommon or when effects are compared across a wide range of a continuous covariate.

The planning target must therefore be explicit. An overall treatment contrast, a treatment contrast within one subgroup, and a difference between subgroup treatment contrasts are different quantities with different precisions. A non-significant estimate in one subgroup is not, by itself, evidence that treatment effects are equal across subgroups.

Effects, Weights, and Precision

The overall effect is a prevalence-weighted average, not an unweighted average unless the subgroups are equally prevalent. Its precision depends on how many participants contribute information to each arm and subgroup. The interaction is a contrast of two treatment contrasts, so its variance includes uncertainty from all four treatment-by-subgroup cells.

For a simple planning calculation, assume a two-arm randomized trial, independent participants, a continuous endpoint with common residual standard deviation \(\sigma\) in all cells, and no clustering. Assume treatment allocation is balanced within each subgroup and that the subgroup proportions in the analysis population are \(q\) and \(1-q\). If each cell has sample size \(n_{ag}\), where \(a\) indexes treatment and \(g\) indexes subgroup, then the estimated cell-mean contrast has variance \(\sigma^2(1/n_{ag}+1/n_{cg})\), where \(c\) denotes control. This calculation presumes the endpoint and analysis population are appropriate for the estimand; attrition or other design features require corresponding adjustments to the effective cell sizes or variance assumptions.

The second equality uses independent estimates from disjoint subgroups. More generally, the variance of a difference includes a covariance term, \(\operatorname{Var}(\hat{\tau}_1-\hat{\tau}_0)=\operatorname{Var}(\hat{\tau}_1)+\operatorname{Var}(\hat{\tau}_0)-2\operatorname{Cov}(\hat{\tau}_1,\hat{\tau}_0)\); the covariance is zero in this simple independent-cell setting. Model-based estimates may have a different covariance structure, so their standard errors should come from the fitted model rather than mechanically applying this formula.

Why Interactions Require Larger Samples

With total sample size \(N\), balanced allocation to treatment and control, and subgroup prevalence \(q\), the expected cell sizes are \(qN/2\) in each arm of subgroup 1 and \((1-q)N/2\) in each arm of subgroup 0. Substituting these sizes into the interaction variance gives a direct planning relationship.

This derivation shows why imbalance in subgroup prevalence reduces interaction precision: the product \(q(1-q)\) is largest at \(q=0.5\) and falls as the subgroup becomes rarer. The same total sample size then supplies fewer observations to at least one of the contrasts needed to form the interaction. Precision also worsens as endpoint variability increases.

Continuous Covariates and the Distribution of Information

Heterogeneity need not be represented by a binary subgroup. With a continuous baseline covariate, a linear interaction model can describe how the treatment contrast changes across the covariate. In a model \(Y=\beta_0+\beta_T T+\beta_X X+\beta_{TX}TX+\varepsilon\), \(T\) is coded 0 for control and 1 for treatment, and \(X\) is centered at its target-population mean. The treatment contrast at covariate value \(x\) is \(\beta_T+\beta_{TX}x\); \(\beta_{TX}\) is the change in that contrast for a one-unit increase in \(X\).

For balanced arms, independent errors with common variance \(\sigma^2\), and the same covariate distribution in both arms, the estimated interaction slope is the difference between treatment-arm and control-arm covariate slopes. If \(S_{XX,T}\) and \(S_{XX,C}\) are the within-arm sums of squared centered covariate values, its variance is \(\sigma^2(1/S_{XX,T}+1/S_{XX,C})\). More spread in \(X\) generally increases these sums and improves slope precision. A large sample concentrated in a narrow covariate range may therefore estimate the overall effect well but estimate variation across that range poorly.

Interpretation, Diagnostics, and Alternatives

Before interpreting an interaction estimate, check the design and model assumptions that support its standard error. Verify the actual treatment-by-subgroup counts, the subgroup prevalence in the analysis population, endpoint variability within cells, and whether allocation was balanced as planned. Sparse cells, strong variance differences, clustering, or substantial departures from the assumed analysis population can make the simple formulas unreliable. For continuous covariates, inspect the observed range and distribution in each arm and assess whether the chosen functional form is plausible.

A frequent error is to compare subgroup p-values: “the treatment effect is significant in subgroup 1 but not subgroup 0, so treatment works differently.” This reasoning is invalid because a difference between significance decisions is not itself a significant difference between effects. The relevant estimate is \(\hat{\delta}=\hat{\tau}_1-\hat{\tau}_0\), with its own standard error and interval. Even a non-significant interaction may be too imprecise to rule out clinically important heterogeneity.

The simple cell-mean approach is transparent for a prespecified binary subgroup and common-variance continuous endpoint. A regression interaction model can incorporate covariate adjustment and continuous modifiers, but its interpretation and precision depend on the model specification and covariate distribution. Prespecified stratification can help ensure adequate representation of a key subgroup, but does not create information without sufficient participants. More complex models may borrow information across groups, but their conclusions depend on additional assumptions; they do not make sparse subgroup data intrinsically precise.

Subgroup analyses are also vulnerable to multiple comparisons, data-driven cut points, and selective emphasis on extreme estimates. Prespecify clinically motivated modifiers and the scale on which heterogeneity is assessed, report estimates with intervals, and distinguish exploratory patterns from confirmatory claims. The treatment contrast can vary on one scale and appear more or less uniform on another, so the endpoint scale is part of the scientific question.