From Stage Results to Claims
A seamless phase 2/3 design does not produce one undifferentiated conclusion. Phase 2 results can determine whether the trial stops, which treatment proceeds, or which population is studied. Each possible action changes what phase 3 will test—and therefore what the final evidence can support. A decision map makes those connections explicit before outcomes are known.
The map is not just a flowchart of operational choices. For every route through it, it should identify the treatment contrast, the population, the phase 3 test, and the claim that a successful test would justify. It should also make clear which claims are not licensed by that route. The earlier material on defining the objectives of each stage distinguishes the roles of screening and confirmation; here the emphasis is on connecting those roles without letting a selection decision silently broaden the eventual conclusion.
To make the mapping concrete, suppose the outcome is a continuous clinical score for which larger values are better. Let \(\theta_{T,C,P}\) denote the mean score difference between treatment \(T\) and control \(C\) in population \(P\), with positive values favoring treatment. A superiority test in population \(P\) might test \(H_0:\theta_{T,C,P}=0\) against \(H_a:\theta_{T,C,P}\ne 0\). The map must specify the relevant \(T\), \(C\), and \(P\); a finding about one contrast or population does not automatically establish another.
Examples below assume random allocation within the stated comparison, comparable outcome measurement between groups, independent participants, and a standard error appropriate to the design. For the simple mean-difference calculations, assume the sampling distribution is adequately approximated by a normal distribution. These are hypothetical data. A real confirmatory design must also specify its endpoint, analysis, significance level, and any control of error across multiple claims or decision paths.
What Each Route Must Specify
For every phase 2 outcome that can change the trial, record four linked items: the decision, the confirmatory test, the claim if the test succeeds, and the limits of that claim. A statement such as “advance the promising option” is not enough: “promising” needs an operational rule, and the route must identify what evidence will later count as confirmation.
| Map element | Question to answer | Example |
|---|---|---|
| Phase 2 outcome | What result triggers this route? | The estimated effect for treatment B in the overall population exceeds a prespecified screening threshold. |
| Action | What changes because of that result? | Advance B; discontinue treatment A. |
| Phase 3 test | Which treatment contrast and population are tested? | B versus control among all eligible participants. |
| Potential claim | What conclusion follows if the test succeeds? | B is superior to control in the defined all-comer population on the specified endpoint. |
| Claim boundary | What does the route not establish? | It does not establish superiority for A or a separate effect in a biomarker-defined subgroup. |
A phase 2 signal is a reason to choose a route, not by itself confirmation of the final claim. In particular, the estimate that caused a treatment to advance should not be described as though it were a successful phase 3 test. The map should also cover stopping: a route that ends development has no phase 3 efficacy claim, even if phase 2 estimates look favorable.
Worked Example: Advancing One Treatment for an All-Comer Claim
A hypothetical phase 2 study compares treatments A and B with control. The prespecified screening rule advances a candidate when its estimated mean benefit is at least 2 score points. The observed estimates are 1.4 points for A and 3.1 points for B. The rule therefore advances B and discontinues A. Phase 3 tests B against control in all eligible participants.
In phase 3, the hypothetical estimated mean difference is 5.2 points, with \(SE=1.8\) points. The test statistic is:
For a two-sided normal test, the p-value is approximately \(2[1-\Phi(2.889)]=0.0039\), where \(\Phi\) is the standard normal cumulative distribution function. If the prespecified phase 3 significance level is \(0.05\), this result rejects \(H_0\) for the B-versus-control contrast in the all-comer population. The corresponding unadjusted 95% confidence interval is \(5.2\pm1.96(1.8)\), or approximately 1.67 to 8.73 score points.
The mapped claim is that B is superior to control in the defined all-comer population on this endpoint, subject to the trial’s prespecified analysis and error control. The result does not support a claim that A is superior: A was not the phase 3 comparison. Nor does a positive estimate alone establish that the benefit exceeds a separate clinical-importance threshold. That would require a claim and test aimed at that threshold.
Population Choice Sets a Claim Boundary
A phase 2 decision can select a population as well as a treatment. For example, a biomarker pattern may lead the design to continue only in participants who test positive. The phase 3 claim must then name that population. A successful test within the selected subgroup does not, by itself, establish benefit in all eligible participants. Conversely, an all-comer test does not automatically establish a confirmatory claim for every subgroup.
A useful way to check this boundary is to write the claim before seeing phase 3 results in the form: “Treatment ___ is superior to control for population ___ on endpoint ___.” Then compare each blank with the phase 3 protocol and the decision-map branch. If a branch changes the treatment, population, or intended conclusion, the test and its error control must cover that route.
Worked Example: A Subgroup Route Supports a Subgroup Claim
Suppose a hypothetical phase 2 study observes an estimated benefit of 6.0 score points for treatment B among biomarker-positive participants and 0.8 points among biomarker-negative participants. The prespecified rule advances B only in the biomarker-positive population. A separate phase 3 comparison of B with control in that population estimates a mean difference of 4.6 points with \(SE=1.5\) points.
The phase 3 statistic is:
The two-sided normal p-value is approximately \(2[1-\Phi(3.067)]=0.0022\). If this is the sole confirmatory claim and the prespecified two-sided level is \(0.05\), the result supports superiority of B over control for biomarker-positive participants on the specified endpoint. Its unadjusted 95% confidence interval is \(4.6\pm1.96(1.5)\), or approximately 1.66 to 7.54 score points.
The claim remains population-specific. The phase 2 estimate for biomarker-negative participants does not turn the phase 3 subgroup result into an all-comer claim. To support a broader claim, the design would need an appropriate phase 3 population and inferential plan for that broader question. The decision map should not allow a subgroup route to be reported as though it had tested all eligible participants.
Why the Whole Map Matters for Error Control
A confirmatory conclusion is selected from a set of possible conclusions by the design’s decision rules. If several treatments, populations, or claims can reach phase 3, looking only at the test on the route ultimately taken can miss the overall false-positive risk. The relevant family is determined by the claims the adaptive procedure permits, not merely by the number of p-values printed in a final report.
A simple justification shows why the map matters. Let \(R_j\) be the event that the \(j\)th false claim is declared, among \(m\) possible claims. The probability of at least one false claim is bounded by:
This inequality means that controlling each possible claim at a nominal level does not automatically control the probability of any false claim at that same level. A prespecified plan may use a valid multiplicity procedure or another design-specific approach. The decision map identifies the possible claims that such a plan must address; it does not, on its own, supply the adjustment.
For example, if the map permits both an all-comer claim and a biomarker-positive claim, the protocol should say whether both are confirmatory and how error is controlled across them. If the biomarker-positive route is mutually exclusive with the all-comer route, the design still needs a prespecified inferential strategy for the adaptive selection. The correct strategy depends on the design and data used; one should not infer validity merely from the fact that phase 3 follows phase 2.
Worked Example: Two Possible Claims and a Prespecified Error Allocation
Consider a hypothetical plan that permits two confirmatory claims for treatment B: superiority to control among all eligible participants and superiority among biomarker-positive participants. For illustration, the plan uses a Bonferroni allocation of a two-sided familywise level of \(0.05\), assigning \(0.025\) to each claim. The phase 3 results are an all-comer estimated difference of 3.2 points with \(SE=1.6\), and a biomarker-positive estimated difference of 5.4 points with \(SE=2.0\).
For the all-comer claim, \(z=3.2/1.6=2.000\), giving a two-sided normal p-value of approximately 0.0455. Because \(0.0455>0.025\), this claim does not meet its allocated criterion. For the biomarker-positive claim, \(z=5.4/2.0=2.700\), giving a two-sided p-value of approximately 0.0069. Because \(0.0069<0.025\), that claim meets its allocated criterion.
Under this illustrative plan, the results support the biomarker-positive claim, but not the all-comer claim. The Bonferroni bound follows from the union inequality above: if each of two tests has type I error at most \(0.025\), the probability of at least one false rejection is at most \(0.025+0.025=0.05\). This calculation assumes the specified tests are valid for their claims; it does not resolve other design issues or replace the need to prespecify how adaptive treatment or population selection is handled.
Diagnostics, Alternatives, and Limitations
A decision map is going wrong when its branches cannot be translated into clear hypotheses, when the phase 3 population differs from the population named in the proposed claim, or when reports make claims for treatments or groups that the selected route did not test. Other warning signs include a transition rule based on vague terms such as “promising,” a confirmatory claim that is broader than the phase 3 enrollment, and multiple possible claims with no stated error-control plan.
A practical review can trace each branch from trigger to conclusion:
Define which phase 2 result or rule activates the branch; avoid relying on an informal retrospective label such as “the better treatment.”
Record whether the trial stops, advances a treatment, changes the target population, or permits more than one option to proceed.
Specify the treatment contrast and population for each phase 3 test, then identify the claim a successful test would support.
List all claims reachable under the map and verify that the protocol’s inferential plan covers the adaptive choices and any multiplicity.
One alternative is to use phase 2 only for screening, then conduct a new, independent confirmatory comparison for the selected treatment and population. This can make the claim mapping easier to explain, but the design still needs a prespecified selection rule and appropriate testing plan. Another approach is to test several treatments or populations confirmatorily, which may retain more options but creates a broader set of claims and an associated error-control problem. A seamless design can share a protocol and transition between stages, but that operational continuity does not make the phase 2 screening result confirmatory.
The map also has limits. It clarifies what each route is intended to establish, but it cannot repair an unsuitable outcome, an ill-defined target population, a poorly implemented randomization, or an analysis that does not match the planned question. The treatment contrast and population must remain interpretable through the transition, and details of the estimand and its preservation require separate specification. A map is therefore a design and reporting discipline, not a substitute for the statistical design itself.
Check Your Understanding
Answer each question by tracing the decision, test, and claim. Brief solutions follow.
- Phase 2 advances treatment A for biomarker-positive participants. Phase 3 tests A against control only in that subgroup and rejects its null hypothesis. What population claim is supported?
- A phase 2 rule selects B over A. Phase 3 tests only B against control. Can the result establish that B is superior to A?
- A confirmatory plan permits three claims, each tested at level \(0.05\). What is the general concern, and what simple bound would result if each test has type I error at most \(0.05\)?
- In a two-claim Bonferroni plan with familywise level \(0.05\), an observed p-value is \(0.031\). Does that claim meet its allocated criterion?
Solutions. (1) It supports a claim about A versus control in the biomarker-positive population, not an all-comer claim. (2) No. The phase 3 contrast is B versus control; superiority of B over A was not tested. (3) The chance of at least one false claim may exceed \(0.05\). The union bound is at most \(0.05+0.05+0.05=0.15\), so a suitable multiplicity strategy is needed. (4) No. Each claim is allocated \(0.05/2=0.025\), and \(0.031>0.025\).