Subgroup Treatment Effects Under Missingness Assumptions

Why Subgroup Effects Need Their Own Sensitivity Analysis

An overall treatment effect can conceal different effects among clinically meaningful groups. But subgroup estimates can also be unstable, especially when outcomes are frequently missing or a subgroup contains few participants. The relevant question is therefore not just whether subgroup estimates differ in one analysis. It is whether that difference is estimated with useful precision and remains similar under the same plausible assumptions about missing outcomes.

The comparison must use shared sensitivity assumptions: each scenario applies the same assumptions about missingness to every subgroup. This makes the subgroup contrast interpretable as a comparison under one coherent analysis, rather than a contrast assembled from separately chosen assumptions. Shared assumptions do not imply equal treatment effects across subgroups; they specify how missing outcomes are handled in each group.

The estimand and its scale must be stated. Here, outcomes are measured in their original units, and a positive \(\tau_g\) means a higher mean outcome under treatment than control in subgroup \(g\). A different outcome scale, such as a risk difference or log odds ratio, defines a different effect-modification contrast.

A valid interpretation requires that subgroups be defined in a way that supports the intended comparison, ordinarily using baseline characteristics rather than outcomes observed after treatment. The treatment comparison also needs the usual design or identification conditions within subgroups: for example, randomization, adequate treatment representation in each subgroup, and valid outcome measurement for observed outcomes. Missing outcomes require additional assumptions. The sensitivity analysis does not make those assumptions true; it shows what estimates follow if each stated assumption is used.

Estimating Effects Under a Shared Scenario

For each subgroup and treatment arm, begin with an observed-data estimate and apply the adjustment implied by the chosen missingness assumption. The adjustment may depend on the observed outcome distribution, missingness pattern, and sensitivity parameters. The exact calculation depends on the model used earlier in the analysis; there is no universal adjustment formula. The essential requirement here is to apply one common scenario \(\eta\) across all subgroups and then calculate each subgroup’s treatment contrast.

For example, suppose a specified scenario adds an arm- and subgroup-specific shift \(d_{ag}(\eta)\) to the observed-data mean estimate \(\hat m_{ag}\). Then

The shifts need not be numerically identical across subgroups: they may act on different observed data or missingness patterns. “Shared” means that the same specified assumption or parameter values govern the adjustments, not that the resulting subgroup adjustments must match. If a scenario requires subgroup-specific departures, those values and their relationship must be stated; otherwise, a comparison can inadvertently give different groups different assumptions.

Comparing Subgroups Without Comparing Separate P-Values

The direct quantity for effect modification is the difference between subgroup treatment effects, \(\Delta(\eta)\), not whether one subgroup’s estimate is statistically significant while another’s is not. Separate significance tests answer whether each subgroup effect differs from zero under its own sampling model. They do not test whether the subgroup effects differ from each other.

For a fixed scenario, a confidence interval for each subgroup effect describes sampling uncertainty conditional on that scenario. A confidence interval for \(\Delta(\eta)\) describes uncertainty in the subgroup contrast, also conditional on that scenario. If subgroup estimates are independent, an appropriate standard error is \(SE\{\hat\Delta(\eta)\}=\sqrt{SE\{\hat\tau_1(\eta)\}^2+SE\{\hat\tau_0(\eta)\}^2}\). If they are not independent, the covariance must be included:

Disjoint participant subgroups in a randomized study may yield independent estimates under suitable analysis conditions, but independence should not be assumed automatically. Shared estimated quantities, clustering, or an analysis that couples subgroup estimates can induce covariance. Use the covariance structure appropriate to the estimator. A normal-approximation interval has the form estimate \(\pm 1.96\times SE\); other justified interval procedures may be needed for small samples or non-normal estimators.

Does Apparent Effect Modification Persist?

Calculate \(\hat\tau_g(\eta)\) and \(\hat\Delta(\eta)\) at each shared scenario of interest. Then describe both the pattern of point estimates and their conditional uncertainty. A subgroup difference that changes direction or size substantially across plausible scenarios is sensitive to missingness assumptions. A difference that retains a similar direction and clinically relevant magnitude across those scenarios is more stable to the assumptions examined, though it can still be imprecise.

This comparison should not be reduced to counting how many scenario-specific intervals exclude zero. Instead, report the estimated contrast, its uncertainty, and how the estimates move over the sensitivity set. A collection of conditional intervals is not automatically a single interval with coverage across all assumptions. In particular, an interval calculated at one scenario does not include uncertainty about the choice of scenario.

Sparse Subgroups, Diagnostics, and Limits

Sparse subgroups deserve particular caution. Few participants, few observed outcomes, or very few participants in one treatment arm can make subgroup means and standard errors unstable. Missingness can further reduce the information available for the arm-specific estimates. A large-looking subgroup contrast may therefore reflect sampling variation, sensitivity to a small number of observations, or the assumed missing-outcome adjustments rather than reliable biological or clinical effect modification.

Useful diagnostics include checking, for every subgroup and arm, the numbers randomized, the numbers with observed outcomes, and the amount of missingness; inspecting whether estimates or standard errors are unusually unstable; and confirming that all groups use the same scenario definitions. Also check that the fitted analysis can estimate the requested quantities within each subgroup and that no adjusted mean is being interpreted beyond the support of the model or data. Extreme shifts or very large changes in an estimate across small changes in assumptions are warnings to report, not reasons to hide a scenario.

A subgroup-specific analysis is not a remedy for weak information. Nor does a shared sensitivity scenario remove the need to justify its assumptions. Results apply to the selected subgroup definitions, outcome scale, model, and sensitivity set. They do not automatically generalize to unexamined subgroups or establish a mechanism for a difference.

The main alternatives answer different questions. A pooled treatment-effect estimate summarizes an average effect and can be useful when subgroup differences are not the target, but it can conceal heterogeneity. Separate subgroup estimates show the effects within groups, but without their direct contrast they do not quantify effect modification. A model with treatment-by-subgroup terms can estimate the interaction directly and may borrow information through its structure; it still depends on correct model specification and must be paired with a missingness sensitivity analysis. None of these approaches makes sparse subgroup evidence precise by itself.