Why the Comparison Must Match the Enrichment Decision
When an interim review leads a trial to continue enrollment only in a biomarker-defined population, the target of the continuing trial changes. The question is no longer necessarily whether treatment benefits every enrolled participant. It may instead be whether treatment benefits participants who meet the selected population definition. A sound answer depends on comparing treatment groups that were randomized within that population, not on comparing groups that merely look similar.
The key practical distinction is between selecting who may enroll and assigning treatment among those who enroll. Enrichment changes eligibility; it does not itself create a randomized treatment contrast. To preserve a direct randomized comparison, eligible participants in the selected population must continue to be assigned to treatment and control concurrently under a specified randomization procedure. The control group must not be discarded simply because enrollment has narrowed.
For a continuous outcome \(Y\), one common target is the difference in mean outcomes, treatment minus control, among eligible participants in the selected population. For a binary outcome, it may be the difference in response proportions. The direction of benefit depends on the outcome scale: for a symptom score where lower is better, a negative difference may favor treatment; for a recovery score where higher is better, a positive difference may favor treatment.
The target should be stated before choosing the calculation. It is usually an intention-to-treat contrast: participants are analyzed according to randomized assignment, regardless of whether they adhered to assigned treatment. The population definition must also be operational, such as a specified biomarker threshold measured before the relevant randomization. Excluding participants after randomization based on their outcomes or adherence can undermine the randomized comparison.
Conditions for a Valid Post-Selection Contrast
The following conditions support a direct randomized comparison after enrichment. They do not, by themselves, guarantee that every aspect of an adaptive trial has controlled error rates; the testing plan must also account for any outcome-based selection and multiple hypotheses as specified in the trial design.
- Eligibility is applied consistently. The selected-population rule is defined and applied before the continuing-stage randomization. The same rule governs treatment and control participants.
- Randomization continues within that population. Treatment assignment is generated by the planned procedure among eligible participants, possibly with stratification or blocking. Allocation concealment and implementation should be maintained.
- There is a concurrent control group. Control participants are enrolled in the same continuing period and under the same eligibility and follow-up procedures as treated participants.
- The analysis preserves assignment. The primary contrast follows randomized assignment, and outcome ascertainment and missing-data handling do not selectively favor one arm.
- The chosen inference matches the outcome and assumptions. For example, a two-sample mean comparison requires independent participants across arms; a model-based \(t\) procedure also relies on its distributional or large-sample conditions.
These conditions matter particularly when the population was selected using interim treatment-outcome evidence. Conditional on that selection, the interim evidence may be unusually favorable for the selected population. A new randomized comparison formed from independently randomized participants recruited after the decision does not reuse those interim outcomes. Under a null of no treatment effect and the randomization assumptions, its treatment assignments remain valid for a test of the selected-population treatment contrast.
Worked Examples
Worked Example: A Continuous Outcome in the Selected Population
Suppose a hypothetical trial selects biomarker-positive participants at interim and then enrolls only biomarker-positive participants. During the continuing stage, 24 are randomized to treatment and 24 to control. At the prespecified visit, the treatment group has a mean improvement of 11.2 points with sample standard deviation 5.4 points; the control group has a mean improvement of 8.1 points with sample standard deviation 4.8 points. Assume larger improvement is better.
Because participants were randomized concurrently within the selected population, the difference in sample means estimates the treatment-minus-control mean contrast for that population:
Use a two-sided Welch \(t\) test for \(H_0\): equal population means versus \(H_a\): unequal population means. Its standard error is
Thus \(t=3.10/1.475=2.102\). Welch’s degrees of freedom are
The two-sided \(p\)-value is approximately 0.041. At \(\alpha=0.05\), this test rejects equal means under the stated model. A 95% confidence interval is approximately \(3.10\pm2.014(1.475)=[0.130,6.07]\) points. In this hypothetical example, the interval estimates a treatment benefit of about 0.13 to 6.07 improvement points in the selected population. The interval and test rely on independent observations, appropriate outcome measurement, and the Welch procedure’s suitability; randomization alone does not correct missing outcomes or major protocol deviations.
Worked Example: A Binary Outcome After Enrichment
In a second hypothetical continuing stage, 40 participants in the selected population are randomized to each arm. A response is observed in 28 treatment participants and 18 control participants. Assume each participant contributes one binary response and randomization and follow-up are conducted comparably between arms.
The observed response proportions are \(\hat{p}_T=28/40=0.700\) and \(\hat{p}_C=18/40=0.450\). The estimated treatment-minus-control difference is \(0.700-0.450=0.250\), or 25.0 percentage points. To test equality of response probabilities, the pooled estimate under \(H_0\) is
The null standard error for the two-proportion score test is
The test statistic is \(z=0.250/0.1105=2.262\), giving a two-sided \(p\)-value of approximately 0.0237. The pooled expected numbers of responses and nonresponses per arm under the null are 23 and 17, respectively, so the usual normal approximation is reasonable for this illustration. At the 0.05 level, the data provide evidence of different response probabilities in the selected population. This test is based on a concurrent randomized comparison; it does not establish that the same difference applies outside that population.
Worked Example: Why Naively Pooling the Interim Stage Can Mislead
Consider a hypothetical trial in which the interim review selects the biomarker-positive population after seeing unusually favorable interim treatment results in that group. Among the 12 biomarker-positive participants per arm at interim, the mean improvement is 12.0 points for treatment and 7.0 for control. In the continuing stage, 16 participants per arm are randomized within the selected population; their means are 8.8 and 7.1 points, respectively. Larger improvement is better.
The continuing-stage randomized contrast is \(8.8-7.1=1.70\) points. A pooled calculation gives treatment mean \((12(12.0)+16(8.8))/28=284.8/28=10.171\) points and control mean \((12(7.0)+16(7.1))/28=197.6/28=7.057\) points. The pooled difference is therefore \(10.171-7.057=3.114\) points.
The pooled value is not automatically a valid ordinary test estimate conditional on the interim selection. The interim observations helped choose the population precisely because their observed contrast was favorable; treating them as if the population had been fixed independently of those outcomes ignores that selection. The continuing-stage contrast avoids reusing those selected interim outcomes and is the direct randomized comparison for this stage. A prespecified selection-aware method may use information from both stages, but simply pooling the means and applying a standard fixed-population test is not a substitute for that method.
Why the Post-Selection Randomization Works
The justification can be stated by conditioning on the interim information. Let \(D_1\) denote all interim data, and let the selection rule choose a population \(S\) from \(D_1\). Suppose that, after this decision, eligible participants are independently recruited and randomized within \(S\), with treatment assignment following the planned randomization. Under the sharp null hypothesis that assignment has no effect on any participant’s outcome, the continuing-stage outcomes are unaffected by their randomized labels.
provided the continuing-stage allocation is generated as planned and is not altered in response to interim outcome values beyond the specified eligibility decision. Thus, conditional on the selected population and interim data, treatment labels in the continuing stage remain governed by the randomization procedure. Under the sharp null, a randomization test that permutes labels only as allowed by that procedure has its stated null distribution. This conditional argument explains why the selection does not itself invalidate a new, independent randomized comparison.
For model-based inference about a mean difference rather than a sharp null, further assumptions are needed. In the Welch example, the observations are independent across participants, the outcome is measured on a common scale, and the sampling distributions are sufficiently close to normal for the \(t\) approximation to be reliable; with small samples, severe skewness or outliers may make that approximation poor. Randomization supports causal interpretation of assignment in the trial population, subject to consistency, no interference between participants, and adequate outcome ascertainment. It does not guarantee representativeness beyond those enrolled.
Comparing Analysis Choices and Diagnosing Problems
| Approach | What it compares | Main concern or use |
|---|---|---|
| Continuing-stage randomized contrast | Concurrent treatment and control participants eligible for the selected population | Direct comparison protected by the continuing randomization |
| Naive pooled contrast | Interim and continuing participants combined by arm | Can ignore outcome-dependent selection; requires an appropriate selection-aware plan |
| Historical-control comparison | Continuing treatment participants versus earlier or external controls | Not protected by concurrent randomization; time trends and case mix may confound the contrast |
| Within-arm pre/post change | Each arm’s outcome compared with its own baseline | Does not replace the randomized between-arm treatment contrast |
Before interpreting a result, check the randomization records, eligibility logs, and timing of biomarker measurement. Confirm that participants in both arms met the same selection rule and that treatment allocation was maintained within any randomization strata. Tabulate enrollment, follow-up, and missing outcomes by arm. A large difference in withdrawal or outcome availability can compromise the comparison even when initial randomization was sound; report the extent and pattern of missingness and use the prespecified approach.
Also inspect whether treatment and control were enrolled in the same calendar period and assessed on the same schedule. If the selected population was defined differently after the interim decision, or if control allocation stopped while treatment enrollment continued, the intended randomized contrast may no longer exist. Do not repair such problems by treating historical controls as if they had been concurrently randomized.
Common Pitfalls and Limits
- Assuming the subgroup label guarantees randomization. A biomarker subgroup is not itself a randomized comparison. Verify that treatment was randomized among eligible participants in that subgroup.
- Using all interim data without accounting for selection. The interim population was chosen using data. Ordinary pooled inference may not retain its nominal error properties conditional on that choice.
- Dropping concurrent controls. A treatment-only continuation cannot estimate a randomized treatment-versus-control contrast from that stage.
- Overgeneralizing the result. The contrast estimates an effect in the selected population under the trial conditions; it does not automatically estimate the effect in all-comers or a broader clinical population.
- Confusing statistical validity with precision. A correctly randomized comparison can still have a wide interval or low power, especially if the selected population is small or recruitment slows after enrichment.
A direct randomized contrast after enrichment is powerful because it preserves a clear comparator while changing the eligibility population. Its limits are equally important: it addresses the population actually randomized, and it cannot on its own establish effect differences between subgroups or solve selection-adjusted combination of stages. Those are distinct inferential questions. For this tutorial’s purpose, the central diagnostic question is simple: were treatment and control participants randomized concurrently under the same selected-population rule?
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- After enrichment, why is a concurrent randomized control group needed to estimate the treatment effect in the selected population?
- A continuing-stage trial observes 32 responses among 50 treatment participants and 24 among 50 controls. Find the two observed response proportions and their difference, treatment minus control.
- In a post-selection trial, treatment and control means are 6.4 and 5.2 points, with standard errors for the two means of 0.50 and 0.60 points. Calculate the standard error of their difference, assuming independent groups.
- Why might a naive analysis combining interim and continuing-stage observations be misleading if interim outcomes helped select the population?
Solutions. (1) Concurrent randomization makes treatment assignment the planned source of the arm difference among eligible participants; historical or treatment-only comparisons can reflect calendar-time or case-mix differences. (2) The proportions are \(32/50=0.640\) and \(24/50=0.480\). Their difference is \(0.640-0.480=0.160\), or 16.0 percentage points. (3) For independent group means, \(SE=\sqrt{0.50^2+0.60^2}=\sqrt{0.61}=0.781\) points. (4) Selection based on interim outcomes can make those observed effects unusually favorable in the selected population. A standard pooled test that treats the population as prespecified ignores that data-dependent selection; combining stages requires an analysis designed to account for it.