Why a Familiar Cutoff May Not Be the Design Cutoff
A normal critical value such as 1.96 is familiar because it gives a specified tail probability for one test. In a group-sequential design, however, efficacy may be assessed at several planned looks. The overall type I error is the probability of crossing the efficacy boundary at any of those looks when the null hypothesis is true. It is therefore not generally the same as the tail probability attached to a cutoff at one look.
A textbook value can be a useful approximation, a nominal per-look cutoff, or a rounded value from a particular design. Those interpretations are not interchangeable. To interpret a calibrated classical boundary, one must know at least the planned look schedule, the target overall alpha, and whether the design uses an upper one-sided or two-sided rule. The boundary’s shape or family also matters: calibration finds the scale of the boundary for the chosen design; it does not make that boundary universal.
The standard canonical null model assumes that the standardized statistics have mean zero, unit variance, and the joint normal distribution implied by independent information increments. Let \(t_k\) be the information fraction at look \(k\), with \(0<t_1<\cdots<t_K=1\). Under this model, \(Z_1,\ldots,Z_K\) are jointly normal, and their correlation is \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\) when \(i\le j\). This dependence matters: the test statistics at successive looks are not independent repetitions of a test.
In a classical boundary family, the look-specific cutoffs have a selected shape and a scale. For example, a Wang–Tsiatis boundary can be written \(b_k=c\,t_k^{s-1/2}\), where \(s\) sets the shape and \(c\) is the scale. Once the family, shape, look schedule, sidedness, and target alpha are specified, calibration determines \(c\) so the null probability of crossing any boundary is the target. A printed or remembered value for \(c\) need not apply to another schedule or alpha.
Why the Joint Null Probability Is Needed
The probability of crossing at some look is a union probability. To see why adding per-look tail probabilities is not an exact calculation, define \(A_k\) to be the event that the first crossing occurs at look \(k\). These events cannot occur together, so the overall alpha is their sum. But \(P(A_k)\) is not simply the marginal probability \(P(Z_k\ge b_k)\): it also requires that no earlier boundary was crossed. Equivalently, summing the marginal probabilities counts some paths more than once.
For two looks with a common upper cutoff \(c\), and with correlation \(\rho=\sqrt{t_1/t_2}\), the null crossing probability has a particularly direct form. Here \(\Phi_2(c,c;\rho)\) denotes the probability that both standard normal statistics are below \(c\).
This formula accounts for the overlap between the two upper-tail events through their correlation. For more looks, calibration uses the joint null distribution and the full vector of boundaries. In practice, the required probability is evaluated numerically, and a cutoff or scale is found by solving for the target overall alpha.
Worked Example: Checking a Textbook Cutoff at Two Looks
Suppose a hypothetical efficacy design has upper one-sided looks at information fractions \(t_1=0.50\) and \(t_2=1.00\), and uses a common cutoff \(c=2.176\). Under the canonical null model, the correlation is
The single-look upper-tail probability is \(1-\Phi(2.176)\approx0.01477\). If the two look-specific tests were incorrectly treated as independent, the probability of at least one crossing would be \(1-(1-0.01477)^2\approx0.0293\). That is not the appropriate calculation for these correlated statistics.
Using the joint normal probability instead gives \(\Phi_2(2.176,2.176;0.7071)\approx0.9748\). Thus the overall crossing probability is approximately \(1-0.9748=0.0252\), not 0.0250. The cutoff 2.176 is close to a 0.025 design cutoff for this schedule, but it is not calibrated to 0.025 at the stated precision. A common cutoff calibrated to 0.025 for these two looks is approximately 2.179.
The small numerical difference is meaningful when reporting a design’s nominal error control. Rounding a cutoff may be entirely appropriate for presentation, but the report should not imply greater calibration precision than the calculation supports.
Nominal Cutoffs, Conservative Cutoffs, and Calibrated Cutoffs
A nominal cutoff is tied to the tail area for a single look. For instance, a one-sided normal cutoff with upper-tail area 0.025 is approximately 1.960. Applying that same cutoff at two or more looks does not, by itself, make the design’s overall alpha 0.025. A Bonferroni approach instead divides a target alpha across the planned looks and chooses each cutoff to make the sum of the marginal tail probabilities no greater than the target. That controls the overall error by the union bound, but it can be conservative because the crossing events overlap.
Calibration uses the joint null distribution for the specified schedule and boundary. It directly targets the desired overall alpha under the stated model, rather than relying on a single-look tail area or a general upper bound. These are different approaches to describing or controlling error, not different names for the same cutoff.
| Cutoff approach | What it uses | What it tells you |
|---|---|---|
| Nominal single-look cutoff | A tail probability for one statistic at one look | The marginal significance level at that look, not necessarily the design’s overall alpha |
| Bonferroni cutoff | A per-look tail probability no larger than the target alpha divided by the number of looks | A conservative upper bound on overall alpha, without requiring the joint crossing probability for calibration |
| Calibrated classical cutoff | The boundary family, look schedule, sidedness, target alpha, and joint null model | The cutoff or boundary scale that attains the target overall alpha under those specifications |
Worked Example: A Bonferroni Cutoff for Two Looks
For a hypothetical two-look, upper one-sided design with target overall alpha \(0.025\), the Bonferroni allocation assigns at most \(0.025/2=0.0125\) to each look. The standard normal cutoff with upper-tail area \(0.0125\) is approximately 2.241.
For any dependence between the two test statistics, the union bound gives
This is a valid conservative control, not an assertion that the crossing probability is exactly 0.025. In contrast, a calibrated common cutoff uses the actual correlation and is chosen to make the joint crossing probability equal to 0.025. Which is more useful depends on the design task: a simple bound may be convenient, while exact calibration can use the available type I error more closely.
Interpreting a Calibrated Boundary in Context
A calibrated cutoff is meaningful only with its design specifications attached. Suppose a report gives a common upper cutoff of 2.179. That number alone does not identify the overall alpha. For the two-look schedule at information fractions 0.50 and 1.00, it corresponds approximately to an overall one-sided alpha of 0.025 under the canonical null model. A different number of looks, different information fractions, different boundary shape, or a two-sided rule changes the calibration problem.
Nor is the cutoff’s one-look nominal p-value the design’s overall alpha. At \(Z=2.179\), the one-sided normal tail is approximately \(1-\Phi(2.179)=0.0147\). That marginal tail area describes a single standard normal statistic. The overall alpha for the calibrated two-look design is approximately 0.025 because it includes the probability of crossing at either planned look.
Worked Example: Reading the Schedule and the Error Rate Together
Consider two hypothetical upper one-sided designs, both targeting overall alpha \(0.025\). Design A has one planned look at the final information fraction \(t_1=1\). Design B has looks at \(t_1=0.50\) and \(t_2=1.00\), with a common cutoff.
For Design A, there is one opportunity to cross, so the calibrated cutoff is the one-look normal quantile:
For Design B, the two statistics have correlation \(\sqrt{0.50}=0.7071\), so a common cutoff must be calibrated using \(1-\Phi_2(c,c;0.7071)=0.025\). The solution is approximately \(c_B=2.179\). The cutoffs differ even though both designs target the same overall alpha, because Design B includes an additional planned opportunity to cross.
If a hypothetical trial following Design B observes \(Z_1=2.05\) at the interim look and \(Z_2=2.15\) at the final look, neither value reaches 2.179. The efficacy boundary is not crossed. The fact that both values exceed 1.960 does not change that decision: 1.960 is the calibrated one-look cutoff for Design A, not the boundary for Design B. This example illustrates why the schedule must accompany the cutoff when interpreting a result.
Assumptions, Diagnostics, and Limitations
The calibration described here relies on the model used to obtain the joint null distribution. In the canonical formulation, the test statistics are approximately jointly normal under \(H_0\), the information fractions correctly describe their correlation, and the planned analysis times correspond to the specified schedule. The test statistic and direction of the efficacy rule must also match the calibration. If, for example, a design was calibrated for an upper one-sided rule, its cutoff cannot be read as a two-sided cutoff without a corresponding two-sided calculation.
Before interpreting a reported cutoff, check the following:
- Look schedule: Are the number and information fractions of planned looks stated? A cutoff calibrated for one schedule does not automatically transfer to another.
- Target and sidedness: Is the target overall alpha stated, and is the test one-sided or two-sided?
- Boundary specification: Is the cutoff common across looks, or does the selected classical family use different look-specific boundaries?
- Precision: Are the reported cutoffs rounded? A rounded value may give an overall crossing probability slightly above or below the target.
- Model fit to the design: Do the statistic, information measures, and planned analysis times support the joint null model used in calibration?
A useful diagnostic is to recompute the null crossing probability from the reported schedule and boundary, using sufficient numerical precision. If it differs from the stated target more than rounding can explain, check whether the schedule, sidedness, boundary family, or target alpha was transcribed incorrectly. Another warning sign is a cutoff presented without enough design information to reproduce its calibration.
Calibration is not protection against every departure from the assumptions. A change in the actual look schedule, a different statistic, or a poorly represented null distribution can make the stated joint probability inaccurate. Calibrated classical boundaries are also tied to their planned design; they should not be assumed to retain the same error control under unplanned changes. The introductory material on classical boundaries provides the broader design context; the key point here is that calibration is conditional on explicit design specifications.
Common Mistake
Check Your Understanding
Answer each question using the definitions and examples above. Short solutions follow the questions.
- For two planned looks, why is adding the two marginal upper-tail probabilities generally not an exact calculation of overall alpha?
- A two-look design uses a common cutoff of 2.176 at information fractions 0.50 and 1.00. What approximate overall crossing probability was obtained, and is the cutoff calibrated to 0.025 at the reported precision?
- For a two-look design targeting overall alpha 0.025, what is the Bonferroni per-look tail allocation and approximate common cutoff?
- Why can two designs targeting the same overall alpha use different cutoffs?
- A report gives a calibrated cutoff but omits the look schedule and sidedness. What information is missing for interpreting it?
Solutions. (1) The crossing events are dependent and overlap; adding marginal probabilities counts paths that cross at both looks more than once. The exact calculation uses their joint null distribution. (2) The overall crossing probability is approximately 0.0252, so 2.176 is close but not calibrated to 0.025 at that precision; a common cutoff is approximately 2.179. (3) The allocation is \(0.025/2=0.0125\) per look, giving \(\Phi^{-1}(0.9875)\approx2.241\). This gives a conservative bound, not necessarily exact calibration. (4) The probability of crossing depends on the number and timing of looks, the resulting dependence among statistics, and the boundary specification. (5) At minimum, the planned look schedule and whether the rule is one-sided or two-sided are needed; the target alpha, boundary family, and model assumptions are also needed to verify the calibration.