Worked Subgroup Analysis from Estimand to Report

From a Subgroup Question to a Causal Estimand

A subgroup analysis is most informative when its question is specified before examining outcomes. The target subgroup should be defined using a baseline characteristic, the treatment contrast and outcome should be clear, and the scale of comparison should be named. Without those choices, estimates that appear to answer the same question may refer to different populations or different notions of effect.

Consider a hypothetical individually randomized trial of a treatment intended to prevent an adverse outcome. Let \(A=1\) denote assignment to treatment and \(A=0\) assignment to control; let \(Y=1\) indicate that the adverse outcome occurs; and let \(S\) be a baseline risk category, with \(S=1\) for high risk and \(S=0\) for lower risk. The subgroup variable is measured before randomization, so it cannot be changed by treatment assignment.

This estimand is defined for each subgroup in its own target population: participants with \(S=s\). It is an intention-to-treat contrast if \(A\) represents randomized assignment, regardless of whether participants follow their assigned regimen. A relative-scale estimand can also be defined as the subgroup risk ratio \(RR_s=E[Y(1)\mid S=s]/E[Y(0)\mid S=s]\), provided the control risk is positive. A risk ratio below one favors treatment for an adverse outcome.

A causal interpretation requires that the randomized assignment is independent of the potential outcomes within the baseline subgroup, that the observed outcome equals the potential outcome under the assigned treatment (consistency), and that one participant’s assignment does not affect another participant’s outcome (no interference). We also assume the outcomes are observed for all participants in this illustrative data set. In a real trial, missing outcomes require additional assumptions and methods. Randomization supports the within-subgroup comparison because \(S\) is measured at baseline; selecting a subgroup based on a post-treatment variable would not have this justification.

Hypothetical Trial Data and Subgroup Effects

Suppose the trial has 200 participants assigned to each arm within each subgroup. The following table gives the number with the adverse outcome and the corresponding observed risk. All numbers are hypothetical.

Absolute effects are often directly useful for decisions because they describe the change in outcome probability, in percentage points. Relative effects describe proportional change. The scale matters: a treatment can have different subgroup contrasts on the risk-difference scale and on the risk-ratio scale. Any claim of effect modification therefore needs to name its scale.

Uncertainty for Subgroup Effects and Their Contrast

For this worked analysis, use independent binomial sampling within each treatment-by-subgroup cell and a normal approximation for the estimated risk differences. For two independent proportions \(\hat p_1\) and \(\hat p_0\), the estimated variance of their difference is \(\hat p_1(1-\hat p_1)/n_1+\hat p_0(1-\hat p_0)/n_0\). This approximation is used here as a transparent large-sample calculation; other settings may call for different interval or test procedures.

To assess additive effect modification, compare the two subgroup risk differences. Define the additive interaction contrast as \(\delta=\Delta_1-\Delta_0\). It is the difference between the treatment effects on the risk-difference scale, not either subgroup’s treatment effect on its own. Here, the estimated contrast is \(-0.100-(-0.010)=-0.090\), or 9.0 percentage points more reduction in the high-risk subgroup.

The interaction test answers a different question from the two tests of whether each subgroup’s effect equals zero. In particular, finding a statistically significant estimate in one subgroup and a non-significant estimate in another does not itself show that their effects differ. The direct comparison is the interaction contrast and its uncertainty.

Multiplicity and a Cautious Results Report

If subgroup effects are tested repeatedly, the chance of at least one false-positive finding can exceed the nominal significance level. The correction depends on the family of claims that the analysis intends to make. For illustration, suppose the family is defined in advance as the two subgroup-specific tests of \(H_0:\Delta_s=0\), one for each subgroup. A Bonferroni procedure controls the family-wise error rate at 0.05 by testing each two-sided hypothesis at \(0.05/2=0.025\), or by multiplying each unadjusted p-value by two, up to one.

The subgroup-specific Bonferroni result does not overturn the interaction result. The high-risk estimate is statistically distinguishable from zero under this specified family-wise procedure, but the estimated difference between subgroup effects still has a 95% interval that includes zero. A subgroup effect can be distinguishable from zero even when the evidence that it differs from another subgroup’s effect is inconclusive.

A concise report should state the subgroup definition, risks in both arms, the effect scale, uncertainty, and the analysis plan’s multiplicity treatment. It should also distinguish the estimated direction and size of heterogeneity from the strength of evidence for it. For these hypothetical data, an appropriate summary is:

Common Pitfalls

• Equating one significant subgroup result with heterogeneity. A test against zero within one subgroup does not test whether subgroup effects differ. Use the direct contrast \(\Delta_1-\Delta_0\) to assess additive effect modification.
• Reporting a subgroup effect without naming its scale. A 10-percentage-point risk difference and a 33.3% relative risk reduction describe the high-risk estimate differently. Neither is a scale-free summary.
• Treating a non-significant interaction as proof of no difference. The interaction interval here includes both a meaningful additional reduction and a small difference in the opposite direction. Failure to reject zero is not proof of equivalence.
• Ignoring the analysis family. An unadjusted p-value does not account for searching across multiple subgroup hypotheses. State what family was considered and which correction was applied.
• Overstating a randomized subgroup result. Randomization supports causal comparisons within a subgroup defined at baseline, subject to the stated assumptions. It does not make every post hoc pattern reliable or show that subgroup membership itself causes different treatment response.

A disciplined subgroup analysis links each claim to its estimand: report treatment effects within the named baseline populations, compare those effects directly when asking about heterogeneity, and express uncertainty on the same scale as the claim. Statistical evidence should then be interpreted alongside the size and clinical relevance of the estimated effects, rather than as a substitute for them.