Prespecifying Subgroups in Trial Protocols

Why a Subgroup Plan Must Come Before Outcomes

A trial may show an average treatment effect while clinicians still need to know whether the effect differs across patient groups. That question is easy to phrase after results are visible: try a different age threshold, inspect another biomarker, or report whichever subgroup looks most striking. The flexibility is the problem. When subgroup definitions or analysis choices are selected in response to the observed outcomes, the resulting estimates and tests no longer have the same interpretation as a plan fixed in advance.

A protocol-ready plan is more than a list of variables. It specifies who belongs to each subgroup, what causal contrast will be estimated, on what scale, how it will be estimated, and which questions are confirmatory rather than exploratory. It also records decisions that could otherwise be made after seeing the results, such as cut points, category handling, and the primary outcome scale.

Specify the Subgroup and Its Target Contrast

Begin by naming the subgroup variable precisely. State how it is measured, when it is measured, its units, and how it is coded. For a categorical variable, list the categories and their reference or comparison group. For a continuous variable, say whether it remains continuous or is divided into categories. If a cut point is used, give its numerical value, units, rationale, and rule for observations exactly at the boundary. A label such as “older patients” is not a reproducible definition.

Let \(A\) denote randomized treatment assignment, with \(A=1\) for the intervention and \(A=0\) for control. Let \(S\) be a baseline subgroup characteristic, and let \(Y^a\) denote the outcome that would be observed under assignment \(a\). For a subgroup defined by \(S=s\), a common causal estimand is the conditional average treatment effect:

This expression compares the mean potential outcome under intervention with the mean potential outcome under control among people in the specified subgroup. For a binary outcome, the plan must say whether the contrast is, for example, a risk difference or a risk ratio. The estimand should also state the outcome and follow-up time, the treatment conditions being compared, and the target population. If the analysis concerns treatment assignment regardless of adherence, state that intention-to-treat target explicitly.

A subgroup effect is not itself evidence that effects differ across subgroups. To describe additive effect modification between two groups, the relevant contrast is the difference between their treatment effects, such as \(\tau(s_1)-\tau(s_0)\). The plan should distinguish this contrast from simply reporting a treatment effect separately in each group. It should also name the scale: effect modification on the risk-difference scale need not agree with effect modification on the risk-ratio or odds-ratio scale.

Identification and Estimation in a Randomized Trial

For a baseline subgroup in a randomized trial, randomization supports a causal comparison within that subgroup. In precise terms, if assignment is independent of the potential outcomes conditional on \(S\), both treatment arms have positive probability within each target subgroup, and the observed outcome under the assigned treatment equals the corresponding potential outcome, then:

The assumptions are conditional exchangeability from randomization, positivity, and consistency. Positivity requires that each treatment being compared can occur for members of the subgroup. Consistency links the outcome actually observed to the potential outcome under the treatment actually assigned. Randomization may be stratified on the subgroup variable, but stratification does not replace the need to specify the estimand and analysis.

The equality above justifies estimating subgroup-specific causal effects from the randomized-arm outcomes, subject to the stated assumptions and adequate outcome measurement. It does not guarantee that a particular model is correctly specified, that estimates will be precise, or that missing outcomes are harmless. A protocol should describe how the estimand will be estimated and how uncertainty will be quantified. For a categorical subgroup and a binary outcome, for example, one approach is to estimate each arm’s risk within each subgroup and subtract the risks to estimate a risk difference. A regression model with treatment-by-subgroup terms is another possible approach, but the model and the scale of the reported contrasts must match the target estimand.

Choose a Scale and an Analysis Model in Advance

The scale is part of the scientific question, not a cosmetic reporting choice. An absolute difference in event risks answers how many percentage points the risks differ. A risk ratio answers how the risks compare proportionally. An odds ratio compares odds and is not generally interchangeable with a risk ratio. The protocol should identify one primary scale for each confirmatory subgroup estimand and explain why it is useful for the clinical decision. Other scales may be reported as secondary or descriptive, but they should not quietly replace the prespecified primary scale when results are known.

The model should be chosen to estimate the stated contrast. For a continuous outcome, a linear model may directly target conditional mean differences if its mean structure is appropriate. For a binary outcome, a model for the outcome probability or a model followed by marginal standardization can be used to obtain risks and risk differences. A model coefficient is not automatically the desired causal contrast: for example, a logistic-regression interaction coefficient is on the log-odds scale. If the target is a risk difference, the plan should explain how predicted risks will be converted to that scale.

For continuous effect modifiers, prespecify the functional form rather than searching for a convenient split. If the plan uses a linear treatment-by-marker interaction, state the marker’s units, centering value, and the range over which the interaction will be interpreted. If a non-linear relationship is scientifically plausible, specify the planned form or a limited sensitivity analysis in advance. Do not choose the cut point or curve after comparing which version produces the strongest apparent treatment difference.

Separate Confirmatory Questions From Exploratory Work

A useful plan identifies a small set of prioritized subgroup questions and labels each one. For every confirmatory question, record the subgroup definition, outcome and time point, estimand and scale, model or estimator, and the planned uncertainty summary. State the scientific rationale and whether the question concerns an effect within a subgroup or a contrast of effects between subgroups. A large subgroup estimate in one group and a small estimate in another do not, by themselves, show that the effects differ; the contrast between effects is the relevant quantity.

Exploratory analyses may examine additional baseline characteristics or alternative functional forms, but their status should be explicit. Report how those subgroups were defined and identify analyses that were not part of the confirmatory plan. Avoid presenting a data-driven threshold as though it had been chosen in advance. Prespecification improves transparency; it does not turn an underpowered or imprecise analysis into strong evidence.

The protocol or statistical analysis plan should be dated and finalized before access to unblinded outcome comparisons. Where feasible, analysts can work with treatment labels masked while finalizing data rules and code. The plan should also cover missing subgroup measurements, values outside the expected range, and missing outcomes. State whether the analysis will use complete cases, a specified imputation approach, or another method, and identify any assumptions required. The later interpretation depends on these choices just as much as on the subgroup label.

Common Pitfalls and a Protocol Checklist

The most common mistake is to treat prespecification as a label rather than a set of reproducible decisions. Writing “subgroups will be examined” does not define what will be estimated or how. Likewise, a cutoff chosen after viewing outcome patterns is not made confirmatory merely by being described in the final report as clinically plausible. Keep a dated record of the plan and disclose deviations.

• Vague subgroup definitions. Replace “high risk” with a measured baseline variable, units, threshold, and boundary rule.
• Post-outcome cut-point searching. Do not try multiple thresholds and report only the most favorable one as the planned analysis.
• Mixing scales. State whether the primary effect is an absolute difference, relative effect, or another contrast, and interpret estimates on that scale.
• Confusing subgroup estimates with heterogeneity. Prespecify the contrast between subgroup effects if the question is whether the effect differs across groups.
• Overstating exploratory findings. Mark unplanned analyses as exploratory and avoid treating their apparent patterns as confirmation.

A complete protocol entry can be checked by asking: Is the baseline subgroup variable and its coding reproducible? Is the target population and outcome time point explicit? Are the treatment contrast, estimand, and effect scale stated? Does the model or estimator target that estimand? Are confirmatory and exploratory questions distinguished? Are missing data and deviations addressed? Prespecification cannot eliminate uncertainty, but it makes the analysis auditable and limits opportunities for outcome-driven choices. The detailed plan for handling the combined evidence from multiple subgroups and outcomes should also be stated rather than left implicit.