Why Early Stopping Changes the Rejection Rule
An interim analysis can provide sufficiently strong evidence to stop a trial early for efficacy. But a trial that can reject at the interim and can also reject after a second stage has two routes to a positive conclusion. Calibrating only the final-stage combination statistic as if the interim rejection route did not exist can therefore exceed the intended overall significance level.
A combination test summarizes evidence from stagewise p-values. To add early efficacy stopping, the design must specify both an interim stopping boundary and a final boundary for participants who continue. The final boundary is calibrated jointly with the interim rule, so the probability of rejection by either route is controlled under the null.
Model and Rejection Rule
Consider a one-sided test of \(H_0\) against an alternative in the positive direction. At the first look, let \(p_1\) be the stage-one p-value and define \(Z_1=\Phi^{-1}(1-p_1)\), where \(\Phi\) is the standard normal cumulative distribution function. At the second stage, define \(Z_2=\Phi^{-1}(1-p_2)\) from the stage-two p-value. Large values of either \(Z\)-statistic favor the alternative.
For the exact calibration below, assume that under \(H_0\), \(Z_1\) and \(Z_2\) are independent standard normal random variables. Equivalently, the stagewise p-values are independent uniform random variables under the null. In an adaptive design, this requires the stage-two p-value to remain valid conditional on the first-stage data and any permitted design adaptations. For composite \(H_0\), the required validity must hold at every parameter value in the null; exact uniformity gives exact control, while conservative valid p-values can yield conservative control under the corresponding conditional validity conditions.
Choose a one-sided interim efficacy probability \(\alpha_1\), with \(0<\alpha_1<\alpha\), where \(\alpha\) is the desired overall Type I error. Define the interim boundary \(a=\Phi^{-1}(1-\alpha_1)\). Stop and reject \(H_0\) if \(Z_1\geq a\), or equivalently if \(p_1\leq\alpha_1\). Otherwise, continue and calculate the weighted combination statistic
For continuers, reject if \(T\geq c\), where \(c\) is a critical value calibrated for the early-stopping rule. The weights are fixed in this setup; they reflect the planned relative contribution of the two stages. The condition \(w_1^2+w_2^2=1\) makes \(T\) standard normal under the null before accounting for the stopping rule, but it does not by itself make the final cutoff \(c\) correct.
Calibrating the Final Boundary
Under the stated null model, early rejection occurs with probability \(P(Z_1\geq a)=\alpha_1\). Among those who continue, conditional on \(Z_1=z<a\), final rejection occurs when \(Z_2\geq(c-w_1z)/w_2\). Independence and the standard normal distribution of \(Z_2\) therefore give the overall null rejection probability
where \(\phi\) is the standard normal density. The first term is the probability of early rejection; the integral is the probability of continuing and then rejecting. Choose \(c\) to solve \(\alpha(c)=\alpha\). Since the integrand's tail probability decreases as \(c\) increases, \(\alpha(c)\) decreases continuously from 1 to \(\alpha_1\). Thus, when \(\alpha_1<\alpha<1\), there is a unique finite \(c\) that gives the desired overall error probability.
For equal weights \(w_1=w_2=1/\sqrt{2}\), the conditional tail argument simplifies:
The integral can be evaluated numerically. This is not the same as using \(c=\Phi^{-1}(1-\alpha)\): that ordinary combination-test cutoff allocates the full error budget to the combination statistic and does not account for the interim rejection route.
Worked Example: Calibrating an Equal-Weight Rule
Suppose the target one-sided overall Type I error is \(\alpha=0.025\). The design stops for efficacy at \(Z_1\geq2\), so \(\alpha_1=P(Z_1\geq2)\approx0.02275\). Use equal weights, \(w_1=w_2=1/\sqrt{2}\), and find the final boundary.
The integral equation to solve is
Thus, the probability of final rejection among continuers, counted over the full null distribution, must be \(0.025-0.02275=0.00225\). Numerical evaluation of the integral gives a calibrated boundary of approximately \(c=2.590\). At this value the final-rejection contribution is approximately \(0.00225\), so the total is \(0.02275+0.00225=0.02500\), up to rounding.
The interim rule is \(Z_1\geq2\); if it is not crossed, the rule is \(T\geq2.590\). The final boundary is relatively high because most of the overall error budget has already been used by the early efficacy boundary. Rounding the boundary too coarsely can change the error probability, so implementations should retain adequate numerical precision.
Applying the Calibrated Rule
The decision uses the interim result first. If the interim boundary is crossed, the trial stops and the second-stage test is not used for the efficacy decision. If the trial continues, the stage-two p-value is combined with the stage-one p-value using the prespecified weights and compared with the calibrated final boundary. The stage-two p-value must be based on the planned stage-two analysis and remain valid under the adaptations made after the first look.
Worked Example: A Continuing Trial Rejects at the Final Analysis
Use the calibrated design above: stop if \(Z_1\geq2\), and for continuers reject if \(T\geq2.590\), with equal weights. Suppose a hypothetical trial has \(p_1=0.050\) and, after continuing, obtains a valid stage-two p-value \(p_2=0.010\). Assume the stage-two p-value is valid conditional on the first-stage data and the stated null model applies.
First, \(Z_1=\Phi^{-1}(0.950)\approx1.645\), which is below 2. Equivalently, \(p_1=0.050>0.02275\), so the trial continues. The stage-two normal score is \(Z_2=\Phi^{-1}(0.990)\approx2.326\). Therefore,
Since \(2.808>2.590\), the trial rejects \(H_0\) at the final analysis. The conclusion follows from the calibrated combination rule, not from treating either stagewise p-value as the overall p-value.
Worked Example: Why an Ordinary Combination Cutoff Can Mislead
Keep the same design and suppose a hypothetical continuing trial has \(p_1=p_2=0.040\). These p-values are assumed to satisfy the design's validity conditions. Each gives \(Z=\Phi^{-1}(0.960)\approx1.751\). Since \(p_1>0.02275\), the trial continues, and
The calibrated rule does not reject, because \(2.476<2.590\). If one instead used the ordinary one-sided \(0.025\) cutoff, \(\Phi^{-1}(0.975)\approx1.960\), this same combination statistic would appear to reject. That lower cutoff ignores the interim rejection probability. It is not the final boundary for this early-stopping design, and using it would not preserve the specified overall error rate.
Worked Example: Efficacy Stopping at the Interim Look
Again use the interim boundary \(Z_1\geq2\). Suppose a hypothetical interim analysis gives \(p_1=0.015\), so \(Z_1=\Phi^{-1}(0.985)\approx2.170\). Since \(2.170\geq2\), the trial stops and rejects \(H_0\) for efficacy. No stage-two p-value is needed for that decision. Under the design assumptions, the probability of such an early rejection under \(H_0\) is \(P(Z_1\geq2)\approx0.02275\); this is part of, not additional to, the overall \(0.025\) error budget.
Assumptions, Diagnostics, and Alternatives
The calibration protects the null error probability only if the assumptions behind its calculation match the actual design. Before using the rule, verify that the direction of the test is consistent across looks, that p-values are computed with the same one-sided convention, and that the stagewise evidence has the required null distribution. In particular, if the second-stage test or sample size is selected using interim information, its p-value must still be conditionally valid under \(H_0\). A p-value that is valid only without adaptation is not automatically suitable for this combination rule.
A practical diagnostic is to reproduce the null rejection probability from the implementation: calculate the early-rejection probability, then integrate or simulate the final-rejection probability among continuers, and add the two. Also check the weight normalization, the interim inequality, and the numerical solution for \(c\). If the computed sum differs materially from \(\alpha\), investigate mismatched p-value tails, incorrect weights, an omitted early-rejection contribution, or a stage-two p-value whose conditional validity fails.
The combination-test approach works with stagewise evidence and can accommodate a suitable change in the second-stage test, provided the stage-two p-value remains valid under the null and the combination rule's conditions are met. A conditional-error approach instead starts from the error probability remaining after the interim data and chooses a later rejection rule that does not exceed that conditional allowance. Group-sequential alpha-spending designs specify boundaries across looks using the joint distribution of the accumulating test statistics. These approaches can be designed to control the same overall error, but their calculation and implementation differ; their cutoffs should not be interchanged without recalibration.
The formula here is for one interim efficacy look and one final combination test. It does not automatically cover multiple interim looks, a futility rule with binding consequences, non-normal or discrete p-values treated as exactly uniform, or arbitrary dependence between stages. Nor does Type I error control establish power, appropriate trial conduct, or validity of the scientific model. Those require separate design and analysis checks.
Common Pitfall
- Wrong reasoning: “The combination statistic is standard normal under the null, so I can use the ordinary overall-\(\alpha\) cutoff at the final analysis and also allow early rejection.”
- Correction: The standard-normal statement describes the unselected combination statistic under the specified null model. The actual rejection rule includes both early rejection and final rejection among continuers. Calibrate \(c\) from their combined probability, as in the integral equation.
- Another warning: If the interim rejection probability is already greater than the target \(\alpha\), no final boundary can undo the excess. Choose an interim boundary with \(\alpha_1<\alpha\) before calibrating the final rule.
Check Your Understanding
Work through the questions, then compare your reasoning with the brief solutions.
- Why is \(w_1^2+w_2^2=1\) not enough to justify using the ordinary one-sided \(0.025\) normal cutoff when early stopping is allowed?
- For \(\alpha=0.025\) and \(\alpha_1=0.02275\), what probability must the final-rejection path contribute under the null?
- In the equal-weight design with interim boundary 2, a continuing trial has \(Z_1=1.80\) and \(Z_2=2.00\). Does it reject under the calibrated boundary \(c=2.590\)?
- What must be checked about the stage-two p-value if its test is chosen after the interim analysis?
- What happens to the overall error control if the interim boundary has \(\alpha_1=0.030\) while the target overall level is \(0.025\)?
Solutions. (1) The normalization gives a standard normal combination statistic under the unselected null model, but the total rejection event also includes interim rejection. The final cutoff must account for both paths. (2) It must contribute \(0.025-0.02275=0.00225\). (3) The interim result continues because \(1.80<2\). The combination is \(T=(1.80+2.00)/\sqrt{2}\approx2.687\), which exceeds \(2.590\), so the rule rejects. (4) The stage-two p-value must be valid under \(H_0\) conditional on the interim data and the design adaptation. (5) Early rejection alone has null probability \(0.030\), already exceeding \(0.025\); no final boundary can reduce that probability, so the design cannot meet the stated overall level with that interim rule.