Why the Transition Decision Matters
In a seamless Phase 2/3 trial, the stage boundary can be a planned operational point or a point at which interim evidence determines whether the trial continues. These choices may lead to the same practical outcome—enrolling patients into a later stage—but they do not necessarily lead to the same statistical analysis. If the interim data influence whether the trial continues, which treatment is carried forward, or how many later-stage patients are enrolled, the confirmatory analysis must account for that data dependence.
A transition rule is not made statistically harmless merely by writing it down before the trial. Prespecification makes the rule transparent and reproducible; it does not, by itself, ensure that a later test has its advertised Type I error rate. The key question is whether the evidence used to make the transition decision is also used, directly or indirectly, in the confirmatory inference.
Fixed Decisions and Their Conditions
A transition is fixed in the statistical sense when its occurrence does not select the trial’s path according to the interim efficacy evidence. For example, the protocol might move to the next stage after a specified number of participants have completed follow-up. A delay caused by a fixed administrative calendar can also be independent of efficacy outcomes, provided that the delay or transition is not itself influenced by those outcomes.
When the transition is independent of the relevant outcomes, a planned confirmatory test can generally use its stated calibration, subject to the test’s other assumptions and the prespecified analysis plan. This is not a claim that every analysis of a fixed-transition trial is automatically valid. The population, estimand, endpoint, treatment comparison, and use of data across stages must still match the assumptions of the chosen procedure. In particular, fixing the transition does not justify an analysis that was not calibrated for reusing earlier data.
In contrast, suppose continuation occurs only when interim results are favorable. Among trials that continue, the observed interim results are no longer a representative sample of all interim results under the null hypothesis. If those same results enter the final test, the test statistic’s null distribution after selection need not be the distribution used to calculate its nominal p-value or critical value.
Worked Example: A Fixed Calendar Transition
Consider a hypothetical trial with two stages. The protocol specifies that Stage 2 opens once 120 Stage 1 participants have completed the primary endpoint assessment, regardless of the interim treatment comparison. There is one planned confirmatory test at the end, using a prespecified test calibrated at \(\alpha=0.025\).
The transition rule depends on completion of a fixed number of assessments, not on whether the treatment appears effective. If the timing criterion is applied as written and is not changed in response to efficacy results, it does not select trials for Stage 2 on the basis of those results. Thus the transition itself does not require changing the test’s null calibration.
For this conclusion to apply, the confirmatory procedure must still match the data actually analyzed. For example, if the final analysis includes Stage 1 observations, that use must be covered by the prespecified procedure and its assumptions. The fixed calendar rule alone does not establish that pooled observations can be analyzed with an ordinary single-stage test.
How Data Dependence Enters Error Control
Let \(A\) denote the event that the trial continues to the confirmatory stage, and let \(R\) denote rejection of \(H_0\). If the trial cannot reject after stopping, then under \(H_0\):
This identity follows because rejection requires continuation: \(R\) is a subset of \(A\). It shows why the transition rule and the test after transition must be considered together. If continuation is independent of the relevant data and the test is properly calibrated, the usual test calibration may remain appropriate. If continuation depends on interim outcomes, the conditional distribution of data given \(A\) can differ from its unconditional null distribution.
The equation also clarifies an important nuance: data-dependent continuation does not necessarily inflate Type I error. For example, a valid test using only new, independent Stage 2 data can retain its stated conditional Type I error after a Stage 1-based transition decision. But a test that reuses the outcomes that selected continuation cannot simply assume the same calibration. The appropriate error rate depends on the complete decision-and-analysis procedure, not on the transition rule viewed in isolation.
Worked Examples of Data-Dependent Transitions
Worked Example: Continuing Only After a Favorable Interim Result
Suppose a hypothetical trial calculates a standardized interim statistic \(Z_1\), and continues only if \(Z_1>1.00\). Under \(H_0\), assume \(Z_1\sim N(0,1)\). If the trial continues, a team proposes to combine \(Z_1\) with an independent Stage 2 statistic \(Z_2\sim N(0,1)\) using \(Z=(Z_1+Z_2)/\sqrt{2}\), then compare \(Z\) with the usual one-sided 0.025 critical value, 1.96.
For hypothetical observed values \(Z_1=1.40\) and \(Z_2=1.45\), the transition rule is met because \(1.40>1.00\). The combined statistic is:
Since \(2.015>1.96\), this proposed rule rejects \(H_0\). However, the usual 1.96 calibration for a standard normal statistic does not, by itself, establish a 0.025 Type I error rate for this selected procedure. Under \(H_0\), the analysis is performed only when \(Z_1>1.00\), and \(Z_1\) also contributes to \(Z\). The null distribution of the analyzed statistic conditional on continuation is therefore not simply the unconditional distribution assumed by the ordinary cutoff. A valid confirmatory analysis needs calibration for the transition and the data used in the test.
Worked Example: A Stage 2-Only Test After an Interim Decision
Now consider a hypothetical design with the same continuation rule, \(Z_1>1.00\), but with a confirmatory test based only on new Stage 2 data. Assume under \(H_0\) that \(Z_2\sim N(0,1)\) and is independent of \(Z_1\). The trial rejects if it continues and \(Z_2>1.96\), a one-sided test at \(\alpha=0.025\).
Under the null, the chance of continuation is \(P(Z_1>1.00)\approx0.1587\). Because \(Z_2\) is independent of the continuation decision, its conditional rejection probability remains \(P(Z_2>1.96)=0.025\). Thus the unconditional probability of rejection is:
This is below 0.025. The Stage 2-only test remains a level-0.025 test conditional on continuing, and stopping prevents rejection. This example does not mean that every Stage 2-only procedure is suitable for every confirmatory claim: the hypothesis, selected treatment, and multiplicity across possible choices still need to be addressed. It does show why dependence on interim outcomes does not automatically imply Type I error inflation.
Worked Example: A Transition Based on an Operational Criterion
A hypothetical protocol opens Stage 2 when a central laboratory has processed all 100 planned Stage 1 samples. The laboratory processing schedule was set in advance and is not changed based on treatment outcomes. The final analysis uses a separately defined Stage 2 endpoint comparison at one-sided \(\alpha=0.025\), with no Stage 1 outcomes included in that test.
The transition condition does not select continuation according to the efficacy results. If Stage 2 outcomes are collected independently of Stage 1 outcomes and the test’s model assumptions hold, the transition does not change the null distribution of the Stage 2-only statistic. Its nominal calibration can therefore be retained, subject to any other design features that affect the hypothesis or analysis.
The conclusion would need reconsideration if laboratory processing were accelerated selectively after promising results, if the Stage 2 population or treatment were chosen using Stage 1 outcomes, or if Stage 1 observations were added to the confirmatory test. Those changes would make the actual procedure different from the one described.
A Practical Check for Selection Effects
Before interpreting a confirmatory result, trace the information flow from interim review to final analysis. The relevant question is not only whether the trial continued, but also whether interim evidence determined the treatment, dose, population, sample size, endpoint, or analysis included in the eventual claim.
Specify whether continuation is triggered by a fixed calendar, operational condition, efficacy outcome, or a combination of information sources.
Identify which interim outcomes determine the transition and whether any of those outcomes also enter the confirmatory statistic or affect the claim.
Verify that the confirmatory test controls Type I error for the complete procedure, including the transition and any selection, rather than only for an unselected final analysis.
A prespecified data-dependent rule is preferable to an improvised decision because it can be evaluated before results are known. It still needs a valid error-control argument. Conversely, an outcome-independent transition can simplify that argument, but only when the rest of the design and analysis preserve the conditions required by the planned test.
Comparison, Diagnostics, and Limitations
| Feature | Fixed transition | Data-dependent transition |
|---|---|---|
| What triggers continuation? | A schedule or criterion independent of relevant interim efficacy outcomes | Interim outcomes determine whether or how the trial continues |
| Effect on the analyzed data | Does not select a path by efficacy evidence, if implemented as planned | Can select favorable outcomes or a particular treatment, dose, or population |
| Confirmatory requirement | Use a test valid for the planned data and estimand | Validate the test for the full selection-and-analysis procedure |
| Common diagnostic concern | Operational decisions may have become outcome-informed in practice | The nominal null distribution may not apply after selection |
Useful diagnostics include checking whether decision records show access to interim efficacy results, whether actual transition dates or criteria departed from protocol, and whether the analysis used data that influenced continuation. A mismatch between the prespecified rule and the implemented rule is especially important: the error-control argument must describe what was actually done.
There are several limitations to these broad principles. Independence must be justified, not assumed because data came from different stages; shared participants, shared outcome information, or adaptation of the later-stage design can undermine it. A transition based on an operational measure may also become indirectly outcome-dependent if that measure is affected by treatment response or is used as a proxy for efficacy. Finally, controlling Type I error does not guarantee adequate power, an unbiased effect estimate, or a clinically relevant claim.
Common Mistake
Correction: Prespecification does not make an outcome-based decision independent of the outcomes. Determine whether interim evidence selects the path or enters the final analysis, then check that the complete procedure has the required null calibration. A data-dependent transition may be compatible with valid confirmatory inference, but the test must be valid for that design.
Check Your Understanding
Answer each question, then compare with the brief solutions below.
- A trial opens Stage 2 after a prespecified number of endpoint assessments, regardless of treatment results. Is that transition fixed or data-dependent in the statistical sense?
- Why does writing a response-based continuation rule in the protocol not, by itself, establish Type I error control?
- Under \(H_0\), continuation has probability 0.20. A valid independent Stage 2-only test rejects with conditional probability 0.025 after continuation. What is the unconditional rejection probability if stopping cannot lead to rejection?
- A team continues only after favorable interim evidence and then reuses that evidence in an ordinary final test. What calibration question must be answered?
Solutions: (1) Fixed, provided the assessment-count rule is not changed in response to efficacy outcomes. (2) Prespecification makes the decision rule known in advance but does not remove its dependence on the interim data; selection can change the null distribution of reused data. (3) \(0.20\times0.025=0.005\), or 0.5%. (4) The team must establish that the complete selection-and-analysis procedure—not merely the unselected final test—controls Type I error at the intended level.