Why Nested-Population Rules Need Explicit Fallbacks
An adaptive enrichment trial may consider several eligible populations, such as all eligible patients, patients positive on a biomarker, and a still narrower biomarker-defined subgroup. These populations are nested: every patient in the narrowest group belongs to each broader group. At an interim analysis, the treatment effect may look promising in one population but not another. A prespecified rule must therefore say not only when eligibility narrows, but also what happens when a narrow population does not meet its continuation criterion.
A fallback is not an assumption that the broader population has the same treatment effect as its narrower subgroup. It is a planned option that is considered only using the broader population’s own estimate and criterion. The rule should be written before examining interim treatment results, with the eligible populations, effect direction, thresholds, data-quality checks, and action for every possible outcome specified.
For illustration, suppose the outcome is a benefit-oriented measure, so larger values favor treatment, and define \(\theta_j\) as the treatment-minus-control mean difference in \(P_j\). If lower outcomes are beneficial, reverse the sign or define the effect in a benefit-oriented direction before setting thresholds. The effect and estimand must have the same meaning across the candidate populations, except for the population to which each applies.
Assumptions and a Prespecified Selection Rule
The rule is meaningful only if population membership and the estimands are well defined. For example, biomarker status should be determined by the prespecified assay and cut points, measured before treatment can affect it, and handled consistently across the trial. Patients in each population should have been randomized in a way that supports the treatment comparison there. Interim outcomes must be sufficiently complete and comparable between treatment arms for the planned estimates to be interpretable. If these conditions fail, an apparent difference between population estimates may reflect measurement or data-collection problems rather than treatment-effect heterogeneity.
A simple operational rule uses population-specific minimum-effect thresholds \(\delta_j\), specified in outcome units, and a data-quality condition \(Q_j\). For example, \(Q_j\) may require acceptable assay performance and adequate outcome follow-up under a separately prespecified standard. Define population \(P_j\) as passing when both its estimate meets its threshold and its quality condition is satisfied.
The selection procedure considers \(P_2\), then \(P_1\), then \(P_0\). It selects the first population with \(A_j=1\); if none passes, it stops for lack of a population meeting the continuation rule. Thus the selected population is the narrowest passing candidate, not automatically the narrowest candidate and not automatically the broadest. Thresholds may differ across populations because the clinical relevance of an effect, the estimand, or the planned use of the result may differ. They must be justified and fixed before interim results are examined.
Confirm that the biomarker, outcome, and follow-up data needed for each candidate population meet the protocol’s quality requirements.
Compare its population-specific interim estimate with its own threshold. If it passes, select it under the narrowest-passing rule.
If a candidate fails, assess the next broader population against its own threshold and conditions. Do not carry the narrower estimate forward as though it were the broader estimate.
Record that the continuation rule selected no population. Do not invent a new cut point or threshold after seeing the interim results.
This is a selection rule for trial conduct, not a proof that a treatment effect exists in the selected population. In particular, meeting a point-estimate threshold does not by itself establish statistical significance, control the error rate of a later claim, or make an estimate unbiased after data-dependent selection. The inferential plan for any final claims must address those issues separately.
Why the Broad-Population Effect Can Differ
A broader population contains patients in additional biomarker strata, whose treatment effects may differ. For a mean-difference estimand, the broad-population effect is a weighted average of the effects in its component strata, using the target population’s stratum proportions. This identity explains why a broader fallback must be evaluated on its own terms.
Here \(q\) is the proportion of \(P_1\) in \(P_2\), and \(\theta_{1\setminus 2}\) is the treatment effect among members of \(P_1\) who are not in \(P_2\). The identity follows by writing the mean outcome under each treatment in \(P_1\) as a weighted average of the corresponding stratum means, then subtracting the control mean from the treatment mean. The same reasoning gives the effect in \(P_0\) as a weighted average of the effects in \(P_1\) and \(P_0\setminus P_1\).
The weights must correspond to the target population in the estimand. Interim sample proportions can differ from target proportions through random variation, differential missingness, or recruitment patterns. Using unadjusted interim proportions without considering the estimand can therefore yield a misleading broad-population estimate. When the planned estimand uses standardization or another specified weighting scheme, use that scheme consistently.
Worked Example: Selecting a Narrower Population With a Fallback
Suppose \(P_2\) is a high-biomarker subgroup, \(P_1\) is the biomarker-positive population, and \(P_0\) is the broad eligible population. The protocol specifies thresholds \(\delta_2=5.5\), \(\delta_1=4.2\), and \(\delta_0=2.8\) outcome points. All three quality conditions are satisfied. Interim estimates are \(\hat{\theta}_2=5.0\), \(\hat{\theta}_1=4.5\), and \(\hat{\theta}_0=3.1\) points. Apply the narrowest-passing rule.
For \(P_2\), \(5.0 < 5.5\), so it fails its own threshold. The rule therefore considers \(P_1\), where \(4.5 \geq 4.2\), so \(P_1\) passes. Although \(P_0\) also passes because \(3.1 \geq 2.8\), the rule selects \(P_1\), the narrowest passing population. This conclusion follows from the prespecified order; it is not a judgment that the broad estimate is invalid.
If \(P_2\) had met its threshold and quality condition, it would have been selected first, even if a broader population also passed. If \(P_1\) had failed as well, the rule would have proceeded to \(P_0\). Each comparison uses the estimate and threshold for that specific population.
Worked Example: Falling Back to the Broadest Population
Consider another hypothetical interim analysis. The estimates are \(\hat{\theta}_2=4.6\), \(\hat{\theta}_1=4.7\), and \(\hat{\theta}_0=3.8\) points. The protocol thresholds are \(\delta_2=5.0\), \(\delta_1=5.2\), and \(\delta_0=3.5\), and all quality conditions are satisfied.
The narrowest population fails because \(4.6 < 5.0\). The next population also fails because \(4.7 < 5.2\). The broader population passes because \(3.8 \geq 3.5\). The rule therefore selects \(P_0\). This is a genuine fallback: the broad estimate meets its own continuation criterion even though both narrower estimates do not.
There is no contradiction in the broader estimate being lower than the narrower estimates, or in it passing a different threshold. The thresholds express the protocol’s criteria for the separate population-specific estimands. A fallback is not justified merely because the broader population contains more patients; it is justified here because its own prespecified criterion is met.
Worked Example: Calculating a Broader Effect From Nested Strata
Suppose \(P_2\) makes up \(40\%\) of \(P_1\). The estimated effect in \(P_2\) is 8 points, while the estimated effect in \(P_1\setminus P_2\) is 2 points. Using the target-population weights, the estimated effect for \(P_1\) is
Suppose \(P_1\) makes up \(60\%\) of \(P_0\), and the estimated effect in \(P_0\setminus P_1\) is 1 point. Then
The narrow-subgroup estimate of 8 points cannot be substituted for either broader estimate. The additional strata change the weighted average. These calculations assume the same outcome scale and treatment-effect definition in every stratum, and weights that represent the target populations. In a real analysis, the estimates and weights would need to be obtained under the protocol’s specified estimation procedure.
Diagnostics, Alternatives, and Limitations
A rule should be checked for operational clarity before the trial begins and for data integrity at the interim. Useful checks include whether biomarker classification is available for the patients needed to define each population, whether outcome follow-up is comparable across arms and strata, and whether the estimates use the intended population weights. A sharp change in the estimated effect across nested populations can be clinically meaningful, but it can also signal sparse data, a data-processing error, or an unstable interim estimate. The rule should specify how such issues are reviewed without allowing an ad hoc change to the cut points or thresholds.
The simple point-estimate rule above is transparent, but it does not account explicitly for uncertainty in \(\hat{\theta}_j\). A design can add prespecified uncertainty or information safeguards, provided their purpose and operational consequences are clear. Any such safeguard should be applied consistently and evaluated in advance; adding it after interim results are known changes the decision rule. Small samples, immature outcomes, assay misclassification, and missing biomarker measurements can all make an interim estimate less dependable. These are reasons to examine operating characteristics and data quality, not reasons to choose a more favorable population after the fact.
The rule also depends on a meaningful nesting structure. If marker definitions overlap without forming subsets, the narrowest-to-broadest sequence is not a valid description of the candidate populations. Likewise, a continuous biomarker cut point selected after inspecting interim treatment effects is not equivalent to a prespecified nested-population rule. The candidate definitions and their ordering must be clinically and operationally interpretable.
| Approach | Decision principle | Main trade-off |
|---|---|---|
| Narrowest-passing with fallback | Try candidates in a prespecified narrow-to-broad order; select the first that meets its own criterion. | Allows a planned broader continuation, but requires separate thresholds and estimands for each population. |
| Strict enrichment | Continue only in the narrow subgroup; if it fails, stop. | Simple and focused, but gives up broader continuation even when the broad population meets a useful criterion. |
| Fixed broad-population design | Keep the broad eligible population throughout, without interim population selection. | Avoids this population-selection decision, but does not adapt eligibility to interim subgroup patterns. |
A Bayesian decision rule can instead compare expected utilities or posterior probabilities for candidate populations; that is a different decision framework, with its own assumptions and prespecification requirements. Regardless of the approach, a continuation rule does not itself resolve how final claims across populations will be supported. The rule’s most important practical limitation is that a clear, reproducible selection can still select a population on the basis of noisy interim estimates. Simulations under plausible treatment-effect patterns, biomarker prevalence, missingness, and recruitment conditions help reveal how often each population would be selected and how often the trial would stop.
Common Mistake
Wrong reasoning: “The narrow subgroup missed its threshold, but its estimated effect was large, so we can continue in the broader group and assume it has at least the same effect.” This substitutes one population’s estimate for another’s and ignores the effects and proportions in the added strata.
Correction: Evaluate the broader population using its own prespecified estimand, estimate, threshold, and data-quality conditions. If it does not pass, the fallback rule does not authorize continuation in that population. If no candidate passes, follow the prespecified stop action rather than changing the rule.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a narrow-to-broad rule, \(P_2\) fails, \(P_1\) passes, and \(P_0\) passes. Which population is selected, and why?
- In \(P_1\), \(P_2\) comprises \(25\%\) of the target population. The treatment effects are 10 points in \(P_2\) and 2 points in \(P_1\setminus P_2\). Calculate the effect in \(P_1\).
- Both \(P_2\) and \(P_1\) fail their thresholds, while \(P_0\) passes. What does the rule select?
- Why is it incorrect to assume that a broader population has the same effect estimate as its narrower subgroup?
Solutions: (1) Select \(P_1\), because it is the first passing population in the prespecified narrow-to-broad sequence. (2) The weighted effect is \(0.25(10)+0.75(2)=2.5+1.5=4.0\) points. (3) Select \(P_0\), provided its quality condition is satisfied, because it independently meets its own criterion. (4) The broader population contains additional strata that may have different treatment effects; its effect is a weighted average determined by the target-population composition, not the narrow-subgroup effect alone.