Why Count a Null Path Only Once?
A group-sequential trial examines its test statistic at several planned looks. A null path may cross an efficacy cutoff at more than one look if the trial were allowed to continue, but the trial stops when it first crosses. Overall type I error is therefore the probability of rejecting \(H_0\) at any look under \(H_0\), not the sum of separate per-look error rates treated as though each look were a new, independent test.
The useful organizing idea is to assign every rejecting path to exactly one event: the event that its first efficacy crossing occurs at look \(j\). Those events cannot happen together. Their probabilities can therefore be added without counting a path more than once. The calculation depends on the joint null distribution of the statistics across looks; the statistics at different looks are generally not independent.
For this definition, assume a one-sided upper efficacy test: crossing \(c_j\) leads to rejection of \(H_0\), and not crossing it does not. Assume the look schedule and efficacy cutoffs are fixed in advance, and that the joint null distribution of \((Z_1,\ldots,Z_K)\) is specified or can be evaluated. In many applications that distribution is approximated by a joint normal model, but the partition itself does not require normality. If the statistic has a continuous distribution, whether the cutoff is written with \(>\) or \(\ge\) makes no difference to the probability.
The Partition and Its Consequence
The event \(A_j\) requires no efficacy crossing at any earlier look, followed by a crossing at look \(j\). If \(j<k\), a path in \(A_j\) has already crossed at look \(j\), whereas a path in \(A_k\) has not crossed at look \(j\). Thus \(A_j\cap A_k\) is empty. Further, a path that leads to rejection must have a first efficacy crossing at one of the \(K\) looks, so the rejection event is the union of the first-crossing events.
This identity is a probability statement, not an assumption that looks are independent. Each probability \(P_0(A_j)\) is a joint-path probability: it accounts both for reaching look \(j\) without an earlier efficacy crossing and for crossing at look \(j\). The joint null distribution is needed to calculate those probabilities. Adding the marginal probabilities \(P_0(Z_j\ge c_j)\) would generally count paths that cross at multiple looks more than once.
Worked Examples
Worked Example: Three Looks Without Futility Stopping
Consider a hypothetical one-sided design with three planned looks and upper cutoffs \(c_1,c_2,c_3\). Suppose evaluation under the specified null joint distribution gives the following probabilities for the first-crossing events. These are assumed numerical integration results for this example, not probabilities estimated from a real trial.
| First efficacy crossing | Null probability |
|---|---|
| Look 1: \(P_0(A_1)\) | 0.012 |
| Look 2: \(P_0(A_2)\) | 0.009 |
| Look 3: \(P_0(A_3)\) | 0.004 |
The overall type I error is $$ \alpha_{\mathrm{overall}} =P_0(A_1)+P_0(A_2)+P_0(A_3) =0.012+0.009+0.004=0.025. $$
Thus this design has a 0.025 probability of rejecting \(H_0\) under the specified null model. The 0.009 at look 2 is already the probability of crossing there without having crossed at look 1; it is not simply the marginal probability \(P_0(Z_2\ge c_2)\).
Adding Binding Futility Rules
A binding futility rule can stop the trial before a later efficacy crossing is observed. Let \(l_i\) be a lower futility cutoff at look \(i\), with \(l_i<c_i\). Suppose the rule is to stop for futility when \(Z_i\le l_i\), to reject for efficacy when \(Z_i\ge c_i\), and otherwise to continue. Then a path reaching look \(j\) without stopping must have remained between the futility and efficacy cutoffs at every earlier look. The first-efficacy-crossing event becomes
For \(j=1\), there are no earlier looks to check, so \(A_1^{(F)}=\{Z_1\ge c_1\}\). The same disjoint-union argument gives \(P_0(\text{reject }H_0)=\sum_{j=1}^{K}P_0(A_j^{(F)})\). Paths stopped for futility are not in any later first-crossing event, because they cannot reach those later looks under the binding rule.
A futility rule described as nonbinding is treated differently in design calculations: the efficacy boundaries are commonly evaluated without relying on that futility stop to maintain the stated type I error. If the rule is nevertheless implemented and stops a path, it cannot create an additional efficacy rejection; it may reduce the actual probability of rejection. State which convention is being used before calculating or interpreting an overall error rate.
Worked Example: A Binding Futility Rule Removes Later Paths
Suppose a hypothetical three-look design uses binding futility rules. Under its specified null model, joint-path calculation gives these first-efficacy-crossing probabilities after accounting for the requirement that the trial not have stopped for futility:
| First efficacy crossing | Null probability with binding futility |
|---|---|
| Look 1 | 0.010 |
| Look 2, with no earlier stop | 0.007 |
| Look 3, with no earlier stop | 0.003 |
The overall error for the rule as specified is $$ \alpha_{\mathrm{overall}}^{(F)} =0.010+0.007+0.003=0.020. $$
For comparison, suppose the same efficacy cutoffs, evaluated without applying the futility stops, yield first-crossing probabilities \(0.010\), \(0.009\), and \(0.006\). The corresponding total is \(0.010+0.009+0.006=0.025\). The difference, \(0.025-0.020=0.005\), reflects null paths that would have had an opportunity to cross an efficacy cutoff later but were stopped for futility. The calculation does not mean that each futility decision has a 0.005 probability; it is the change in overall rejection probability for these hypothetical rules.
Using the Sum to Check a Design
The null-path identity is also a direct check on a proposed set of boundaries. Once the joint null model and the stopping rule are specified, evaluate each first-crossing probability and add them. If the resulting sum differs from the target \(\alpha\), the boundaries do not attain that target under the calculation used. Numerical integration or simulation may be used to evaluate the probabilities, but the reported sum should be identified as exact under the model, numerically approximated, or estimated by simulation.
Worked Example: Checking a Final-Look Cutoff
Suppose a hypothetical three-look design has no futility stopping. For fixed first- and second-look cutoffs, joint-null calculation gives \(P_0(A_1)=0.012\) and \(P_0(A_2)=0.008\). The target overall error is 0.025, so the final first-crossing contribution must be $$ P_0(A_3)=0.025-0.012-0.008=0.005. $$
Assume numerical evaluation of the specified null model gives \(P_0(A_3)=0.009\) when the final cutoff is \(c_3=2.05\). Then the total is $$ 0.012+0.008+0.009=0.029, $$ which exceeds the target by \(0.029-0.025=0.004\). Suppose the same evaluation gives \(P_0(A_3)=0.005\) at \(c_3=2.41\). With that cutoff, the sum is $$ 0.012+0.008+0.005=0.025. $$
Under these assumed calculations, \(c_3=2.41\) attains the target overall error, whereas \(c_3=2.05\) does not. This is a numerical boundary check, not a claim that the cutoff can be chosen from the target alone: the result depends on the look schedule, earlier cutoffs, stopping rules, and joint null distribution.
Assumptions, Diagnostics, and Alternatives
The sum is only as reliable as the rejection rule and null model used to define its events. Check that the design has a specified set of looks and that the cutoff decisions are represented correctly. In particular, verify whether equality counts as crossing, whether futility is binding, and whether the test is one-sided or two-sided. For continuous statistics, changing a strict inequality to a non-strict one at a cutoff has probability zero; with discrete statistics, the distinction can matter.
When probabilities are calculated numerically, check that the first-crossing events are disjoint in the implementation and that their probabilities are summed only once. For simulation, record the first stopping look for each null replicate and estimate the overall error by the fraction rejected; the event-specific fractions should sum to that fraction, apart from rounding. Monte Carlo error can make the estimated sum differ from the target, so an estimate near the target is not proof of exact calibration. Also check that the assumed null distribution represents the null hypothesis being controlled. For a composite null, calibration must address the relevant null values, often by controlling the largest rejection probability over the null parameter space.
A Bonferroni approach instead controls the probability of any crossing by bounding it with the sum of the marginal crossing probabilities. This can be useful when a suitable joint distribution is unavailable, but the bound may be conservative because marginal crossing events overlap. The first-crossing calculation uses disjoint events and the joint null law to evaluate the actual rejection probability under that model. Neither approach justifies adding nominal per-look \(p\)-value thresholds as if each were an independent type I error allowance.
For a two-sided design, define a rejection at look \(j\) as crossing either the lower or upper efficacy cutoff, and define the first-crossing event as no efficacy crossing at earlier looks followed by either crossing at look \(j\). Those events remain disjoint across looks, so their probabilities sum to the overall two-sided rejection probability. The two tails must both be included in each relevant look’s event probability; summing only the upper-tail contributions would not describe the two-sided test.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a three-look design without futility, state the event that a path first crosses the efficacy cutoff at look 3.
- Suppose the first-crossing probabilities at three looks are 0.006, 0.011, and 0.008. What is the overall type I error under the specified null model?
- At each look, a binding futility rule stops the trial when \(Z_i\le l_i\), where \(l_i<c_i\). What must be true at looks 1 and 2 for a path to be in the first-efficacy-crossing event at look 3?
- Why is the sum of marginal probabilities \(P_0(Z_j\ge c_j)\) generally not the overall type I error?
Solutions. (1) The event is \(\{Z_1<c_1,\ Z_2<c_2,\ Z_3\ge c_3\}\). (2) The disjoint first-crossing probabilities sum to \(0.006+0.011+0.008=0.025\). (3) The trial must continue at both earlier looks, so \(l_1<Z_1<c_1\) and \(l_2<Z_2<c_2\), followed by \(Z_3\ge c_3\). (4) Marginal crossing events can overlap across looks; adding them can count a path more than once. The first-crossing events are disjoint and partition the rejection event.