Why a Population Switch Changes the Claim
A seamless Phase 2/3 trial may begin in a broad population and continue only in a subgroup, such as patients who test positive for a biomarker. The treatment, comparator, and outcome can remain unchanged while the population changes. That change matters: a treatment effect averaged across a broad group is not automatically the treatment effect in the subgroup chosen for Phase 3. Confirmatory evidence must support the claim about the population named in that claim.
The key task is therefore to make the population explicit before interpreting a test. A claim that treatment improves response in all eligible patients is different from a claim that it improves response in biomarker-positive patients. The second claim needs evidence relevant to biomarker-positive patients, together with a testing procedure that controls the relevant error rate after any selection.
A target population needs more detail than a label such as “biomarker-positive.” It should be operationally defined—for example, by eligibility criteria, the assay and cut-off used to classify biomarker status, and any relevant treatment-history restrictions. The outcome and treatment contrast must also be stated. A treatment effect is interpretable only relative to those choices.
Why Broad and Subgroup Effects Can Differ
For a binary outcome, consider the risk difference comparing treatment with control. Suppose a broad population consists of biomarker-positive and biomarker-negative patients. If the biomarker-positive fraction is \(q\) in both randomized treatment groups, then the broad-population risk difference is a weighted average of the subgroup risk differences:
This identity follows by writing each treatment-arm risk as a weighted average of its subgroup risks and subtracting the control risk from the treatment risk. Unless the subgroup effects are equal, the broad effect does not equal either subgroup effect. A different subgroup mix in the Phase 3 target population can also change the broad-population average.
Worked Example: A Small Broad-Population Effect Hides Different Subgroup Effects
In a hypothetical randomized Phase 2 study, 100 patients per arm include 40 biomarker-positive and 60 biomarker-negative patients. Among biomarker-positive patients, 24 of 40 assigned treatment and 12 of 40 assigned control respond. Among biomarker-negative patients, 18 of 60 assigned treatment and 24 of 60 assigned control respond. Calculate the subgroup and broad-population risk differences.
For biomarker-positive patients, the treatment response risk is \(24/40=0.60\), and the control risk is \(12/40=0.30\). Thus
or 30 percentage points. For biomarker-negative patients, the corresponding risks are \(18/60=0.30\) and \(24/60=0.40\), giving
or minus 10 percentage points. In the broad population, there are 42 responses among 100 treated patients and 36 among 100 controls, so the estimated risk difference is \(0.42-0.36=0.06\), or 6 percentage points. The weighted-average check gives \(0.40(0.30)+0.60(-0.10)=0.06\). A 6-point broad-population estimate does not establish a 6-point effect in the biomarker-positive population—or any particular effect there. The subgroup results are relevant to that subgroup, but their confirmatory interpretation still depends on how the subgroup was selected and how multiplicity was handled.
This example also shows why the phrase “the trial showed benefit” is incomplete. It must be clear which population the estimate describes. If the Phase 3 claim concerns a selected subgroup, a broad-population treatment effect cannot be substituted for that subgroup’s effect simply because the subgroup was included in Phase 2.
What Evidence Supports the New Phase 3 Population?
The most direct evidence for a new target population comes from randomized Phase 3 data collected in that population, analyzed using a prespecified outcome and testing procedure. The Phase 3 test should target the subgroup-specific null hypothesis, not a broad-population null that no longer matches the claim. Randomization must support a valid comparison within the target population, and the analysis must respect the trial’s prespecified error control, including any multiplicity created by considering several candidates or claims.
Phase 2 data may help select which population to study, inform planning, or provide supportive evidence. It can contribute to a confirmatory claim only if the inferential method validly tests the same population-specific hypothesis. In particular, a Phase 2 p-value for the broad population is not a p-value for the null hypothesis in the selected subgroup. If several subgroups were considered and the most promising was chosen using their observed treatment effects, the subgroup result also cannot be treated as an unselected test unless the selection process is covered by a valid procedure.
Earlier material on treatment-selection validity and multiplicity provides the relevant framework for selection-adjusted testing. The population-specific point is that adjustment for selecting a treatment or subgroup does not change the identity of the hypothesis: evidence must still be linked to the population the eventual claim names.
Worked Example: A Direct Phase 3 Test in the Selected Population
Suppose a hypothetical Phase 3 stage enrolls biomarker-positive patients only. Of 100 patients assigned treatment, 60 respond; of 100 assigned control, 40 respond. Assume individual randomization within this eligible population, a binary response outcome assessed consistently in both groups, complete outcome ascertainment, and a prespecified two-sided test at \(\alpha=0.05\). Calculate an approximate test statistic and interval for the risk difference.
The estimated risk difference is \(0.60-0.40=0.20\), or 20 percentage points. Under the null of equal response risks, the pooled response estimate is \((60+40)/(100+100)=0.50\). The pooled standard error for the test is
The z statistic is \(0.20/0.07071=2.828\), corresponding to a two-sided p-value of approximately \(0.0047\). The approximate 95% confidence interval uses the unpooled standard error:
so the interval is \(0.20\pm1.96(0.06928)\), or approximately \((0.064,0.336)\). Under the stated assumptions and the prespecified testing plan, these data provide evidence of a positive treatment effect on response risk in the biomarker-positive Phase 3 population. The interval is an approximate 95% confidence interval for that risk difference, not a 95% probability statement about the fixed effect. This result does not, by itself, establish a treatment effect in biomarker-negative patients or in the entire broad population.
When Earlier-Stage Data Can Contribute
Combining stages is appropriate only when the stagewise evidence supports the same null hypothesis and the combination procedure remains valid under the design. For a biomarker-positive claim, Phase 2 evidence must concern the biomarker-positive effect, rather than merely the broad-population effect. Its p-value must also be valid given the selection and multiplicity process. Stagewise independence, or a correctly specified dependence adjustment, is needed for the chosen combination rule.
Worked Example: Combining Evidence for the Same Population-Specific Claim
Consider a hypothetical design in which both stages provide a valid test of the null hypothesis of no treatment effect in biomarker-positive patients. Assume the Phase 2 p-value, \(p_2=0.20\), is valid for that hypothesis after accounting for the design’s selection procedure, the Phase 3 p-value is \(p_3=0.01\), and the stagewise p-values are independent under the null. For illustration, use an equal-weight inverse-normal combination test with a one-sided alternative in the beneficial direction.
Convert the upper-tail p-values to standard-normal statistics: \(z_2=\Phi^{-1}(1-0.20)=0.8416\) and \(z_3=\Phi^{-1}(1-0.01)=2.3263\). The combined statistic is
The corresponding one-sided p-value is approximately \(0.0125\), below the prespecified one-sided level \(0.025\). Thus the combined procedure rejects the null for the biomarker-positive claim. The conclusion depends on all the stated conditions. If \(p_2\) instead tested the broad-population null, combining it with \(p_3\) would not establish the subgroup claim, even if the same combination formula produced a small number.
The crucial issue is not whether Phase 2 and Phase 3 observations can be combined mathematically. It is whether the resulting test answers the scientific question being claimed. If Phase 2 estimates a different population effect, a valid p-value for that effect cannot be relabeled as evidence for the Phase 3 population. Where transport from one population to another is proposed, it requires explicit assumptions that treatment effects are sufficiently stable across the populations, along with a defensible method for accounting for differences in patient mix and relevant effect modifiers. Such assumptions are substantive and should not be hidden inside a pooled analysis.
Checks, Alternatives, and Common Pitfalls
Before making a confirmatory claim after switching populations, check the target population definition, the estimand, and the hypothesis the analysis actually tests. Then verify that the Phase 3 data represent that target and that selection and multiplicity are addressed. Warning signs include a broad-population p-value presented as subgroup evidence, post hoc subgroup boundaries, unexplained changes in eligibility, or a combined result that joins stagewise tests of different null hypotheses.
| Approach | What it can support | Key requirement |
|---|---|---|
| Phase 3 test in the selected population | A claim about that population | Valid randomized comparison and prespecified error control |
| Combination of Phase 2 and Phase 3 evidence | A claim about one population-specific effect | Valid stagewise tests of the same null and a valid combination procedure |
| Transport or extrapolation from a different population | A claim in the target population only under stated assumptions | Defensible assumptions about effect stability and population differences |
A common mistake is to reason, “The Phase 2 broad-population test was significant, and the selected subgroup is contained within that population, so the subgroup claim is confirmed.” The conclusion does not follow: a broad average can be driven by effects in other patients, and a subgroup chosen after looking at outcomes adds selection concerns. The correction is to test the subgroup-specific null with valid subgroup evidence and appropriate error control, or to state a broader claim that matches the evidence.
A population switch can improve the clinical relevance of a trial by focusing on patients for whom treatment may be most useful. Its limitation is that the resulting claim is narrower and requires evidence for that narrower target. Selection evidence can motivate the change, but it does not by itself replace confirmatory evidence in the selected population.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A broad Phase 2 trial estimates a risk difference of 8 percentage points. Can this alone confirm a Phase 3 claim about biomarker-positive patients? Why or why not?
- In a broad population, the biomarker-positive fraction is \(0.25\), with subgroup risk differences \(0.20\) for positive patients and \(-0.04\) for negative patients. Calculate the broad-population risk difference, assuming the subgroup fraction is the same in both randomized arms.
- Phase 2 tested a broad-population null and Phase 3 tested a biomarker-positive null. Is it enough that both p-values are individually valid to combine them for the biomarker-positive claim?
- Name two design or analysis features that help make a Phase 3 claim about a newly selected population credible.
Solutions. (1) No. The broad estimate averages effects across groups and does not, by itself, test the biomarker-positive null; valid subgroup-specific evidence and error control are needed. (2) \(0.25(0.20)+0.75(-0.04)=0.05-0.03=0.02\), or 2 percentage points. That broad effect is not the 20-point subgroup effect. (3) No. The tests concern different null hypotheses, so combining them does not establish the biomarker-positive claim. (4) Examples include a clear, prespecified biomarker definition and eligibility rule; randomization within the target population; a prespecified subgroup-specific analysis; and an error-control procedure that accounts for selection and multiplicity.