Why a Common Cutoff Needs Calibration
At each planned analysis of a group sequential trial, a standardized statistic can provide evidence against the null hypothesis. If the trial stops for efficacy as soon as a statistic is sufficiently large, however, the chance of a false efficacy conclusion is not simply the chance of exceeding a cutoff at one analysis. There are several opportunities to cross the boundary. The overall error depends on both the cutoff and the dependence among the statistics.
Pocock’s constant critical-value boundary uses the same upper cutoff at every planned look. “Constant” describes the cutoff on the standardized-statistic scale; it does not mean that each look has a separate one-sided type I error of \(\alpha\). The cutoff is selected so that the probability of crossing it at least once under the null is the desired overall one-sided type I error.
The boundary addresses efficacy stopping in the specified direction. It does not, by itself, define a futility rule or a rule for other endpoints or hypotheses. The design must also specify the planned looks and the null distribution of the statistics across them.
Null Model and Overall Error Probability
Let \(t_i\) denote the information fraction at look \(i\), with \(0<t_1<\cdots<t_K=1\). Under the usual canonical group sequential model under \(H_0\), the vector \((Z_1,\ldots,Z_K)\) has a joint multivariate normal distribution with mean zero, unit marginal variances, and correlations \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\) for \(i<j\). Thus the dependence is part of the design, not an optional adjustment.
This model is an appropriate basis for calibration when the planned information fractions are correctly specified and the test statistics have the canonical joint normal null distribution, exactly or to a useful approximation. Analyses must use the same direction of evidence, and the rule being assessed must be the stated rule: reject \(H_0\) for efficacy if any planned statistic reaches the upper boundary. If the statistic or covariance structure differs materially from this model, calibration under this model need not give the advertised error rate.
The overall one-sided type I error is the probability, assuming \(H_0\) is true, of at least one efficacy crossing. Equivalently, it is one minus the probability that all statistics remain below the cutoff. If \(\Phi_K(\,\cdot\,;R)\) denotes the cumulative distribution function of the \(K\)-variate standard normal distribution with correlation matrix \(R\), then
Calibration chooses \(c\) to make this probability equal to the target \(\alpha\), using the planned information fractions and hence the corresponding matrix \(R\). The multivariate probability is generally evaluated by numerical integration or a validated group sequential calculation. A cutoff calibrated for one schedule of looks should not automatically be reused for a different schedule.
Why the Formula Controls the Sequential Error
The stopping rule can also be expressed in terms of first crossings. Define \(A_i\) to be the event that the statistic first crosses the boundary at look \(i\): the statistic at look \(i\) reaches \(c\), and all earlier statistics are below \(c\). These events cannot occur together, because a trial cannot have two different first-crossing looks. Their union is exactly the event of any efficacy crossing.
This identity justifies the error-control calculation: it counts each possible false stopping opportunity once, at the look when it first occurs. The multivariate normal expression gives the same probability by taking the complement of the event that every statistic lies below \(c\). Neither calculation treats the looks as independent.
A useful upper bound follows from the union bound. Each look has marginal null distribution \(Z_i\sim N(0,1)\), so
where \(\Phi\) is the standard normal cumulative distribution function. This bound can be useful for checking a calculation, but it generally does not equal the actual crossing probability. Because statistics from nearby looks are positively correlated under the canonical model, their crossing events overlap. The exact design-wide probability accounts for that overlap.
Worked Example: Find the First Crossing
Consider a hypothetical three-look design with a common upper boundary \(c=2.04\). Suppose the observed standardized statistics are \(Z_1=1.72\), \(Z_2=2.11\), and \(Z_3=2.35\). Apply the stopping rule.
At the first look, \(1.72<2.04\), so the boundary is not crossed and the trial continues. At the second look, \(2.11\ge2.04\), so the trial stops for efficacy at look 2. The third value is not used to decide whether the trial stops: after a first crossing, that later analysis is not reached under this rule.
The relevant decision is the first crossing, not the number of observed statistics that would exceed the boundary if all three were calculated. The boundary is applied in standardized-statistic units; the corresponding effect estimate, if reported, has its own units and interpretation.
Worked Example: Marginal Error Is Not Overall Error
In a hypothetical design with three planned looks and \(c=2.04\), each individual statistic has a standard normal null distribution. Its one-look upper-tail probability is approximately \(1-\Phi(2.04)=0.0207\). Adding these marginal probabilities gives the union-bound calculation
This is an upper bound, not an estimate that the actual overall error is 0.0621. The crossing events overlap because the looks are correlated. For illustration, if the planned information fractions are \(0.25, 0.50, 1.00\), the canonical correlations are \(\operatorname{Corr}(Z_1,Z_2)=\sqrt{0.25/0.50}=0.7071\), \(\operatorname{Corr}(Z_1,Z_3)=\sqrt{0.25/1.00}=0.5000\), and \(\operatorname{Corr}(Z_2,Z_3)=\sqrt{0.50/1.00}=0.7071\). The exact error is therefore obtained from the three-variate normal probability with this correlation matrix, not by treating three marginal tail areas as disjoint.
The calculation also explains why a per-look probability cannot be reported as the design-wide type I error. For example, saying “the chance of a false efficacy finding is 0.0207” would describe one marginal analysis, not the possibility of crossing at any of the three looks.
Calibration in Practice
For a specified target \(\alpha\), the constant boundary is found by solving \(1-\Phi_K(c,\ldots,c;R)=\alpha\). This is a one-dimensional numerical root-finding problem once \(K\), the information fractions, and the null covariance model are fixed. In a validated implementation, the reported cutoff is usually rounded for presentation, while the design calculation uses greater precision.
For example, suppose a hypothetical design has three equally spaced information fractions \(t=(0.25,0.50,1.00)\), and a validated multivariate normal calculation reports that a common cutoff of \(c=2.04\) (rounded) gives an overall one-sided crossing probability of approximately \(0.0500\). The design interpretation is that, under the specified null model and stopping rule, the probability of reaching the efficacy boundary at any planned look is approximately 5%. It is not that each look separately has a 5% type I error. This numerical illustration describes a calibration output; the precise cutoff must be obtained using the design’s full-precision inputs.
Worked Example: Check a Proposed Calibration
A hypothetical design targets an overall one-sided \(\alpha=0.05\) over three planned looks. Its information fractions are \(0.25,0.50,1.00\). A validated calculation reports a common boundary \(c=2.04\) to two decimal places and a crossing probability of \(0.0500\) to four decimal places. Check what this output means and whether the marginal tail areas alone verify it.
The reported output means that, under the design’s null joint distribution, \(P_{H_0}(\max_i Z_i\ge c)\approx0.0500\), meeting the stated target to the displayed precision. At a single look, the upper-tail probability is approximately \(1-\Phi(2.04)=0.0207\), so the sum of the three marginal probabilities is \(3(0.0207)=0.0621\). That sum is only the union bound and does not contradict the reported 0.0500 crossing probability.
The verification must use the full correlation matrix, with off-diagonal entries \(0.7071, 0.5000,\) and \(0.7071\), and evaluate the multivariate probability. The rounded cutoff alone is insufficient to reproduce a high-precision calibration; the calculation should use the unrounded cutoff returned by the software.
Assumptions, Diagnostics, and Alternatives
A useful implementation check starts by verifying that the planned information fractions are strictly increasing, end at 1 for the final analysis, and correspond to the analyses that will actually be conducted. The induced correlation matrix should have unit diagonal, be symmetric, and have entries consistent with the canonical formula. The calibrated probability should be recomputed at the unrounded cutoff; changing the number or timing of looks can change the crossing probability.
Important failure modes include using a two-sided cutoff for a one-sided rule, using an information schedule different from the one used in calibration, and treating the statistics as independent. Another error is to apply the common cutoff to statistics whose sign convention is reversed, so that large values no longer represent evidence in the prespecified direction. If variance estimation, missing outcomes, overdispersion, or other features make the canonical null distribution a poor approximation, the assumed covariance and tail probabilities require scrutiny rather than automatic acceptance.
A main alternative for controlling an overall error rate is a Bonferroni allocation: select per-look tail probabilities whose sum is at most the target. This gives a transparent conservative guarantee without needing to calculate the dependence exactly, but it may not use the known joint structure as efficiently. Pocock calibration instead chooses one common cutoff using the joint crossing probability. Other group sequential boundaries can vary by look; their error is likewise assessed by the probability of crossing any boundary, but their shape and allocation across looks are different questions.
The constant boundary is a specified design choice, not a universal cutoff. Its limitations include dependence on the planned analysis schedule and on the validity of the joint null model. If the actual monitoring plan, test statistic, or one-sided direction changes, the original cutoff may no longer provide the intended error control. The design should document the target error, information fractions, covariance assumptions, calculation method, and full-precision cutoff.
Check Your Understanding
Answer each question using the stated stopping rule and null model.
- For two planned looks with a common upper cutoff \(c\), write the event that produces an efficacy rejection.
- Why is \(K\{1-\Phi(c)\}\) generally an upper bound rather than the exact overall error?
- A trial has a common cutoff \(2.04\), and its first two statistics are \(1.80\) and \(2.06\). At which look does the efficacy rule stop the trial?
- If the planned information fractions change, why should the cutoff be recalibrated?
Solutions. (1) Rejection occurs on \(\{Z_1\ge c\}\cup\{Z_2\ge c\}\), or equivalently \(\{\max(Z_1,Z_2)\ge c\}\). (2) The union bound adds marginal tail probabilities and can count overlapping crossing events more than once; the exact probability uses their joint distribution. (3) The first statistic is below the cutoff, while \(2.06\ge2.04\), so the trial stops for efficacy at look 2. (4) The information fractions determine the canonical correlations among looks. Changing them changes the joint crossing probability for a given cutoff, so the cutoff solving the target overall error can change.