Additional Interim Looks in the Confirmatory Stage

Why Add Interim Looks After the Phase Transition?

A seamless trial may use phase 2 evidence to select a treatment and then continue with a confirmatory stage. Once the confirmatory stage is under way, investigators may want to review accumulating evidence before its planned end. An interim look can support early stopping for efficacy, and it may also support stopping for futility. The challenge is to preserve the confirmatory claim’s type I error control while using both the phase-transition evidence and the new confirmatory-stage data.

The key is to keep two tasks distinct. A combination test combines valid stagewise evidence into a statistic for testing the null hypothesis. A group-sequential boundary specifies how strong that statistic must be at each look to justify stopping for efficacy. The combination statistic at a later look includes the earlier stagewise evidence, but statistics at successive looks are correlated. Therefore, comparing every look separately with an unadjusted critical value is not generally valid.

The p-values are oriented so that smaller values support the prespecified efficacy direction. For a two-sided scientific question, the design must specify in advance how directional evidence is represented; one should not choose the favorable direction after inspecting the results and then use a one-sided p-value. The phase-transition p-value must remain valid after treatment selection, as in the earlier material on valid stagewise p-values after treatment selection. A nominal unadjusted p-value from a treatment chosen because it looked best is not automatically a valid \(p_A\).

Assumptions and the Repeated-Look Statistic

For the standard inverse-normal calibration, under the null hypothesis the transformed stagewise values are independent standard normal variables: \(Z_A,Z_{B1},\ldots,Z_{BK}\) are mutually independent and each has distribution \(N(0,1)\). Equivalently, the corresponding p-values are independent uniform random variables under a simple null. In applications, a valid p-value may be conservative rather than exactly uniform; the design’s validity argument must then ensure the resulting test remains level controlled. Independence of confirmatory increments is supported by an appropriate independent-increments construction: each increment uses new information not reused in another increment, and the analysis accounts for the trial’s allocation and data structure.

The stage A contribution is not required to be independent of the selection decision in an informal sense; rather, the input \(p_A\) must itself have the selection-adjusted validity required by the design. Nor can independence be assumed merely because datasets have different labels. Shared observations, repeated use of outcomes, or unaccounted changes to the analysis can break the assumptions. The earlier material on independent increments across trial stages gives the relevant design principle.

Under the stated exact null model, each \(C_k\) is standard normal. Its numerator is a sum of independent, mean-zero normal variables, with variance equal to the sum of squared weights in its denominator. For example, at look \(k\), $$ \operatorname{Var}\left(w_A Z_A+\sum_{j=1}^{k}w_{Bj}Z_{Bj}\right) =w_A^2+\sum_{j=1}^{k}w_{Bj}^2. $$ Dividing by the square root of this positive variance gives a standard normal statistic. This proves the marginal null distribution at each look, but does not make the sequence \(C_1,\ldots,C_K\) independent.

For \(k<\ell\), the covariance of two cumulative statistics is $$ \operatorname{Cov}(C_k,C_\ell)= \frac{w_A^2+\sum_{j=1}^{k}w_{Bj}^2} {\sqrt{\left(w_A^2+\sum_{j=1}^{k}w_{Bj}^2\right) \left(w_A^2+\sum_{j=1}^{\ell}w_{Bj}^2\right)}}. $$ The shared stage A and earlier confirmatory contributions create this correlation. Thus a sequence of individually valid marginal tests can still have an inflated probability of at least one false rejection if each is conducted at the full nominal level.

Calibrating Efficacy Boundaries

Before the confirmatory stage begins, specify the number and timing of looks, the weights, and the efficacy boundaries \(b_1,\ldots,b_K\). The trial stops for efficacy at the first look \(k\) for which \(C_k\ge b_k\). Under the null, the design must ensure $$ P_{H_0}\left(\text{there is some }k\le K\text{ such that }C_k\ge b_k\right)\le \alpha. $$ This probability is the familywise type I error across the repeated looks for the claim under consideration. Boundaries can be calibrated using the joint multivariate normal distribution of the cumulative statistics, or through a prespecified error-spending procedure. The boundaries need not be equal: a design may spend less type I error early and reserve more for the final look.

The boundary calculation must match the actual test process: the fixed weights, look schedule, and any planned stopping rule. A final-look critical value alone does not account for earlier opportunities to reject. Adding or moving a look after seeing accumulating results, or changing weights based on those results without a validity argument, can invalidate the calibration.

Stopping Rules Can Coexist with Combination Testing

A combination test does not require the trial to continue to the final look. At an interim, the statistic combines all evidence available through that look, and the boundary determines whether that evidence is sufficient for an early confirmatory rejection. If the trial continues, the next statistic adds the next independent increment. If the trial stops for efficacy, no later increment is needed for the prespecified rejection rule.

A futility boundary is a separate decision rule. A nonbinding futility rule allows the trial to continue even if the futility threshold is met; it is used to guide stopping but is not enforced when calibrating efficacy error. A binding futility rule requires stopping when specified, but does not itself create a false efficacy rejection if stopping for futility never counts as rejecting \(H_0\). The design must distinguish this from efficacy stopping and state whether futility is binding. The treatment effect may still be uncertain when stopping for futility; the decision is not proof of no effect.

If several selected treatments or primary endpoints can generate claims, the look-wise boundary calibration addresses repeated looks for a given claim, not necessarily multiplicity across claims. The earlier material on multiplicity across candidate treatments and primary endpoints remains relevant: the complete rejection rule must account for every claim that can be made.

A Necessary Weight Check and a Common Pitfall

Weights control how much each stage contributes to the cumulative statistic. They need not all be positive, but the cumulative squared weight in the denominator must be strictly positive at every look where a statistic is calculated. A weight assigned only to a future increment cannot make an earlier look statistic defined.

A common mistake is to say, “Each \(C_k\) is standard normal under the null, so I can compare each one with the ordinary one-sided \(\alpha\) cutoff.” The marginal distribution statement is correct, but the conclusion is not. Because the statistics are correlated and the trial rejects if any look crosses, the type I error is a joint crossing probability. The correction is to calibrate all boundaries together for the planned look sequence, or use a prespecified spending rule that controls the same overall error.

How This Approach Differs From a Conventional Group-Sequential Test

A conventional group-sequential test typically starts with a single confirmatory-stage test statistic and repeatedly examines its accumulating evidence under an explicitly modeled information process. The inverse-normal combination approach instead combines stagewise p-values, including valid evidence from before the phase transition, with evidence from new confirmatory increments. It is useful when the seamless design needs to preserve a valid phase-transition contribution after treatment selection.

Both approaches require prespecified looks and boundary calibration; neither permits repeated unadjusted testing at the nominal level. A combination test can be particularly natural when stagewise p-values are the valid inputs available after adaptation. It does not, however, make arbitrary data reuse harmless or remove the need to justify the stagewise p-values and independence assumptions. Its performance also depends on the chosen weights: a design that gives little weight to a stage may lose power when that stage contains much of the relevant evidence.

Diagnostics should therefore include checking that every planned statistic has a positive denominator, verifying the direction and calculation of each p-value, confirming that each increment uses the intended data, and reproducing the joint null crossing probability for the exact look schedule and weights. Unexpected dependence, altered analyses, or unplanned looks are signs that the stated calibration may no longer apply. The method controls the specified false-rejection probability under its assumptions; it does not guarantee power, unbiased effect estimation after early stopping, or validity for claims outside the planned multiplicity procedure.