One Scenario, Three Design Choices
An enrichment strategy changes who enters a trial, or which population remains eligible after interim data are reviewed. Comparing strategies is useful only if they address the same clinical question, use comparable assumptions, and account for the consequences of selection. A design with high power in a biomarker-positive subgroup may have little power to establish benefit in the full population; a design that permits adaptation may also require a more carefully calibrated final test.
Consider one hypothetical trial of a treatment for a continuous clinical outcome, where a larger value is better. A binary biomarker is measured before randomization. Assume 40% of eligible patients are biomarker-positive, the treatment effect is 0.35 outcome units in that subgroup, and the treatment effect is 0.00 units among biomarker-negative patients. Within each biomarker stratum, outcomes are normally distributed with variance 1 in both treatment arms. Patients are randomized equally between treatment and control within each enrolled population.
For this workshop, the treatment effect is the treatment-minus-control difference in mean outcome. The target population is the population eligible for the trial before biomarker-based recruitment restrictions. We assume the biomarker is measured accurately, its prevalence is stable, outcomes are observed for all randomized patients, and there is no interference between patients. These assumptions make the design comparisons transparent; if they are implausible, the operating-characteristic calculations must reflect more realistic conditions.
Translate the Clinical Scenario Into Effects
The positive-subgroup effect is 0.35 outcome units, while the negative-subgroup effect is 0.00. The all-comer effect is their prevalence-weighted average. This calculation clarifies why the target population matters: an all-comer average can be modest even when the biomarker-positive effect is clinically meaningful.
Here \(q=0.40\) is the biomarker-positive prevalence, \(\Delta_{+}\) is the positive-subgroup treatment effect, and \(\Delta_{-}\) is the negative-subgroup effect. Under these assumptions, the average treatment effect in the full eligible population is 0.14 outcome units. It is not interchangeable with the 0.35-unit effect among biomarker-positive patients.
Worked Example: Set a Broad-to-Narrow Rule
Suppose the first stage enrolls 160 patients from the full eligible population. At the interim analysis, the design will restrict further recruitment to biomarker-positive patients only if the estimated positive-subgroup treatment effect is at least 0.20 outcome units and the estimated negative-subgroup effect is no greater than 0.10. Otherwise, recruitment stops for futility, with no efficacy claim.
Under the assumed prevalence, the expected first-stage subgroup sizes are \(160(0.40)=64\) positive patients and \(160(0.60)=96\) negative patients. With equal allocation, that is approximately 32 positive patients per arm and 48 negative patients per arm. The rule is fully specified before looking at the interim data: it names the populations, estimates, thresholds, and consequence of each outcome.
These expected counts do not guarantee the realized counts. A protocol must state how it handles a materially different subgroup count, missing biomarker results, or a treatment-arm imbalance within a subgroup. The thresholds are illustrative, not universal clinical cutoffs. Their operating characteristics must be evaluated under plausible treatment effects, prevalence values, and outcome variability.
Compare the Recruitment Architectures
For a concrete resource comparison, suppose each strategy has a maximum of 400 randomized participants. The broad-to-narrow design starts with 160 all-comers and, if the enrichment rule is met, enrolls 240 additional biomarker-positive patients. The fixed-enrichment design enrolls 400 positive patients. The parallel-cohort design enrolls 200 all-comers in one cohort and 200 additional positive patients in a separate cohort; no patient is counted in both cohorts. These are different allocations of the same maximum total, not claims that they have equal precision for every estimand.
| Design | Recruitment plan | Primary target in this comparison | Main trade-off |
|---|---|---|---|
| Broad-to-narrow | 160 all-comers, then 240 positive patients if the interim rule is met | Positive subgroup | Can concentrate later recruitment, but data-dependent selection complicates the final test |
| Parallel cohorts | 200 all-comers plus 200 additional positive patients | All-comer and positive-subgroup effects | Provides evidence in both populations, but splits the total sample and requires joint error control for two claims |
| Fixed enrichment | 400 positive patients from the outset | Positive subgroup | Efficient for the subgroup target, but does not directly estimate the all-comer effect |
The table describes possible architectures, not a ranking. The appropriate choice depends on whether the clinical decision concerns the full population, a biomarker subgroup, or both. It also depends on recruitment feasibility, whether the biomarker can be obtained promptly, and the consequences of a false positive or missed benefit.
Worked Example: Compare Approximate Precision and Power
For a balanced two-arm comparison with known variance 1 in each arm and \(n\) total patients in the population being tested, the standard error of the difference in means is \(\sqrt{4/n}\). For a one-sided superiority test at \(\alpha=0.025\), an approximate power under effect \(\Delta\) is \(\Phi(\Delta/SE-1.96)\), where \(\Phi\) is the standard normal cumulative distribution function. This approximation assumes the stated normal model and a fixed sample and analysis population.
In the fixed-enrichment design, all 400 patients are positive, so \(SE=\sqrt{4/400}=0.100\). Under \(\Delta_{+}=0.35\), the standardized effect is \(0.35/0.100=3.50\), giving approximate power \(\Phi(3.50-1.96)=\Phi(1.54)\approx0.938\). The standard error is in outcome units; the standardized effect and power are unitless.
In the parallel positive cohort, \(n=200\), so \(SE=\sqrt{4/200}=0.1414\). If the positive-cohort test uses \(\alpha=0.0125\), its one-sided critical value is approximately 2.241. Approximate power is \(\Phi(0.35/0.1414-2.241)=\Phi(0.234)\approx0.592\). The positive subgroup is the target, but fewer patients are assigned to that cohort and the test uses a stricter threshold.
In the broad-to-narrow design, the expected positive count after successful enrichment is \(64+240=304\). If that were a fixed, nonselected positive-subgroup sample, its standard error would be \(\sqrt{4/304}=0.1147\), and nominal-\(\alpha=0.025\) power would be approximately \(\Phi(0.35/0.1147-1.96)=\Phi(1.09)\approx0.862\). This is only a fixed-path benchmark. It is not the design's actual power, because the decision to enrich depends on interim data and the rule may stop the trial. Actual operating characteristics must account for those paths.
Error Control After Selection
A test's significance level is meaningful only for the complete decision process that generates the final claim. In a fixed-enrichment design with one prespecified positive-subgroup claim and no data-dependent selection, a one-sided test at \(\alpha=0.025\) can control the probability of a false positive at 0.025 under its null model, subject to the test assumptions. In the parallel-cohort design, making either an all-comer or a subgroup claim creates two opportunities for a false positive. One simple valid approach is Bonferroni allocation: test each claim at \(\alpha=0.0125\), so the probability of at least one false rejection is at most \(0.0125+0.0125=0.025\). This bound does not require the tests to be independent.
The broad-to-narrow design presents a different issue. The interim estimates determine whether the positive subgroup continues, so the final data and analysis population are connected to an earlier data-dependent decision. Applying an ordinary fixed-sample test to the selected subgroup as though selection had not occurred does not, by itself, establish the desired type I error control. A valid design must prespecify an inferential procedure that accounts for every allowed interim path and demonstrate that the probability of a false positive is controlled at the target level under the null configurations relevant to the claim. The procedure may use a formally justified adaptive testing method or a simulation-calibrated rule with adequate validation; the method must match the precise selection and testing plan.
Justify the Approximate Power Calculation
Under the stated model, the treatment and control sample means in a subgroup are independent normal random variables with variance \(1/n_T\) and \(1/n_C\), where \(n_T\) and \(n_C\) are their sample sizes. Their difference therefore has variance \(1/n_T+1/n_C\). With balanced allocation and \(n_T=n_C=n/2\), this gives the standard error used above. Under a true effect \(\Delta\), the standardized test statistic has a normal distribution with mean \(\Delta/SE\) and variance 1. A one-sided test rejects when that statistic exceeds \(z_{1-\alpha}\), so its power is the probability of exceeding that threshold.
This justification applies to a prespecified fixed-sample comparison under the model; it does not establish power for a design that selects a population using interim data. The earlier material on simulation operating characteristics is useful for evaluating the full adaptive procedure, because simulation can reproduce recruitment, interim selection, stopping, and final testing together.
Worked Example: Allocate Error for Two Claims
Suppose the parallel-cohort trial plans two one-sided confirmatory claims: treatment superiority in all-comers and treatment superiority in the biomarker-positive cohort. The family-wise type I error target is 0.025, meaning the probability of at least one false rejection among these claims should be no greater than 0.025 under the applicable null configuration.
Using equal Bonferroni allocation gives \(\alpha_1=\alpha_2=0.025/2=0.0125\). For any configuration of true null hypotheses, the probability of at least one false rejection is no greater than the sum of the individual false-rejection probabilities, which is at most \(0.0125+0.0125=0.025\). This control may be conservative, but it is transparent and valid without an independence assumption. The chosen error allocation should be reflected in the power calculations, as it was for the positive-cohort calculation above.
Evaluate the Designs as Complete Procedures
A single power number is not enough to compare enrichment designs. The design's operating characteristics should describe what happens across plausible scenarios, including the assumed scenario and cases where the biomarker has no predictive value or the prevalence differs from 40%. For each scenario, useful summaries include:
- Probability of rejecting each null hypothesis, and probability of at least one false positive under null scenarios.
- Probability of selecting the positive-only path, stopping for futility, or retaining broad recruitment.
- Expected total enrollment, expected number of biomarker-positive participants, and the chance that recruitment reaches its maximum.
- Bias and uncertainty of the estimated treatment effect in the population named by the final claim.
- Performance when prevalence, subgroup effects, outcome variance, or biomarker classification differs from its planning value.
For the hypothetical broad-to-narrow rule, simulate the first 160 patients, apply the thresholds exactly, then simulate the chosen continuation and apply the planned final test. Under the global null, estimate how often the entire process makes a false positive claim; under the stated alternative, estimate the probability of a positive-subgroup claim and the probability of stopping. Repeat the procedure over many simulated trials and retain Monte Carlo uncertainty for each estimated probability. The same scenario grid and outcome assumptions should be applied to the fixed and parallel designs to make comparisons interpretable.
Worked Example: Decide What a Design Comparison Can Support
Suppose a planning simulation of 10,000 hypothetical trials under the positive-subgroup effect of 0.35 reports 8,600 positive-subgroup rejections for a broad-to-narrow procedure, 9,300 for fixed enrichment, and 5,900 for the positive cohort in the parallel design. These figures are illustrative simulation outputs, not results for a real study. Their estimated powers are \(8600/10000=0.860\), \(9300/10000=0.930\), and \(5900/10000=0.590\), respectively.
The pattern is consistent with the fixed-sample benchmarks, but it does not alone establish that one design is preferable. First verify that the adaptive procedure's type I error is controlled in null scenarios, that its simulation implements the prespecified selection and test correctly, and that the comparison uses the same target claim. Then weigh the power difference against recruitment burden and whether the all-comer effect is also needed. A design can have excellent subgroup power and still be unsuitable when the intended decision concerns all eligible patients.
Diagnostics, Alternatives, and Limitations
Several warning signs call for redesign or further simulation: the selected path changes sharply under small changes in the enrichment threshold; positive-subgroup counts are often much lower than expected; type I error exceeds its target in any relevant null scenario; or the reported power omits the probability of stopping or the cost of a stricter error allocation. Sensitivity analysis should vary biomarker prevalence and subgroup effects, rather than treating their planning values as known constants. Recruitment-time trends or measurement error may also change who is classified and enrolled.
The alternatives answer different needs. Fixed enrichment is simplest when the positive subgroup is the sole target and the biomarker is sufficiently established before recruitment. Parallel cohorts preserve contemporaneous evidence in both populations, at the cost of allocating participants across claims and controlling error across them. Broad-to-narrow recruitment can use early all-comer information to guide later recruitment, but the decision rule and its inferential consequences need advance specification and validation. None of these designs turns an observed subgroup difference into proof that the biomarker predicts differential treatment benefit; that conclusion depends on the relevant comparison and uncertainty, not merely on a significant estimate in one subgroup.
A common mistake is to say, “The adaptive trial ends with 304 positive patients, so its positive-subgroup power and type I error are the same as a fixed trial with 304 patients.” The correction is that the 304 count is an expected count conditional on an enrichment path, and that path was chosen using interim data. Its actual power and false-positive probability are properties of the complete rule, including selection, stopping, and final testing, and must be evaluated as such.
Check Your Understanding
Use the assumptions in this tutorial. The questions test both the calculations and the design logic.
- What is the all-comer treatment effect when prevalence is 0.40, the positive-subgroup effect is 0.35 units, and the negative-subgroup effect is 0.00 units?
- For 400 fixed-enrichment patients and known variance 1 in each arm, calculate the standard error and approximate one-sided power at \(\alpha=0.025\).
- Why is the fixed-path power benchmark for the broad-to-narrow design not necessarily its actual power?
- In the parallel-cohort design, why can two tests each conducted at \(\alpha=0.025\) fail to control the probability of any false positive at 0.025?
Solutions. (1) The weighted effect is \((0.40)(0.35)+(0.60)(0.00)=0.14\) outcome units. (2) \(SE=\sqrt{4/400}=0.100\) outcome units; power is \(\Phi(0.35/0.100-1.96)=\Phi(1.54)\approx0.938\). (3) The final subgroup size and analysis population depend on the interim estimates, and the rule may stop the trial; actual power must average over these possible paths. (4) Testing two claims creates two opportunities for a false rejection. Bonferroni allocation of 0.0125 per claim bounds the probability of at least one false positive by \(0.0125+0.0125=0.025\).