Unequal Directional Error Budgets

Why Separate Directional Budgets?

A two-sided group sequential test can stop early for evidence in either direction. In a clinical trial, an upper-boundary crossing might indicate benefit, while a lower-boundary crossing might indicate harm. The two conclusions need not have equal consequences. A design may therefore assign more of its null rejection probability to one direction than the other.

The allocation is about the probability of a false directional conclusion under the null, not about the probability that the treatment is beneficial or harmful. It must also account for all scheduled looks: repeated opportunities to cross a boundary are dependent, so a per-look tail probability is not the same as the cumulative directional error probability.

Here, the upper direction represents benefit and the lower direction represents harm; the sign convention must be defined for the endpoint and treatment contrast before choosing boundaries. For simplicity, the discussion below uses a point null of zero effect and a standardized statistic \(Z_i\) at look \(i\). The upper and lower efficacy limits are \(u_i\) and \(l_i\), respectively, with \(l_i<u_i\). A trial stops for benefit at the first look with \(Z_i\geq u_i\), and for harm at the first look with \(Z_i\leq l_i\). Otherwise it continues, subject to any separately specified stopping rules.

The usual model for the derivation is that, under \(H_0\), the sequential statistics have a joint multivariate normal distribution with mean zero, unit variances, and canonical correlations determined by information. If look \(i\) has information fraction \(t_i\), where \(0<t_1<\cdots<t_K=1\), then

This is a standard large-sample group sequential model. The canonical covariance structure relies on the information-based normal approximation and the planned information fractions being appropriate for the analysis. A different null distribution, an incorrectly specified information schedule, or unplanned changes to the analysis can invalidate the stated error guarantees. The guarantees are for the specified null model; they do not by themselves establish control over every possible composite null without additional conditions.

Directional First-Passage Probabilities

The first-passage qualification matters. A path that crossed an upper efficacy limit at an earlier look has already stopped; it cannot later contribute to the lower-direction probability. Define \(U_i\) as the event of an upper crossing for the first time at look \(i\), and \(L_i\) as a first lower crossing at look \(i\). Across all looks, these events are mutually exclusive: \(U_i\cap L_j=\varnothing\) for every \(i,j\), and events of distinct first stopping looks are also disjoint. Thus

The last equality holds because a trial cannot first stop for both directions. Consequently, if the design is calibrated so that \(P_0(U)=\alpha_U\) and \(P_0(L)=\alpha_L\), its overall null rejection probability is \(\alpha_U+\alpha_L\). If the requirements are upper bounds rather than equalities, the overall probability is at most \(\alpha_U+\alpha_L\).

One way to see how the dependence enters is to condition on the previous look. Under the canonical model,

The mean and variance in this conditional distribution determine the probability of crossing either limit at look \(i\), given the value at look \(i-1\). But that calculation must be integrated only over paths that survived the earlier looks. For example, a path that remains in the continuation interval through look \(i-1\) lies between \(l_{i-1}\) and \(u_{i-1}\). Integrating over all possible previous \(Z\)-values would incorrectly include paths that had already stopped.

Calibrating an Unequal Allocation

To formulate a design, first state the total type I error requirement and how it is divided by direction. For example, if the overall target is \(0.05\), a prespecified allocation might be \(\alpha_U=0.03\) and \(\alpha_L=0.02\). Then choose or solve for the boundary sequences \(u_1,\ldots,u_K\) and \(l_1,\ldots,l_K\) so that the cumulative first-passage probabilities meet those directional targets under \(H_0\). The allocation does not, by itself, determine the boundary shape: different sequences can in principle meet the same cumulative budgets.

For a numerical calibration, define \(f_i(z)\) as the joint null density contribution at \(Z_i=z\) from paths that have survived through look \(i\). At the first look, this contribution is the standard normal density restricted to the continuation interval. At a later look, it is obtained by integrating over surviving values at the preceding look and using the conditional normal density:

Here, \(\phi(z;m,v)\) denotes the normal density at \(z\) with mean \(m\) and variance \(v\), and \({\bf 1}\{\cdot\}\) is 1 when its condition holds and 0 otherwise. This recursion assumes there are no other stopping rules; if additional rules apply, their continuation restrictions must also be imposed at each look. The probability of first crossing upper at look \(i\) is the probability mass among paths that survived through look \(i-1\) and then cross \(u_i\); the corresponding lower mass is defined in the same way. Summing these look-specific crossing probabilities gives \(P_0(U)\) and \(P_0(L)\). A boundary solver can adjust the limits until those sums meet the allocated budgets.

Interpretation, Comparisons, and Checks

An unequal allocation is useful when the design team wants different false-positive tolerances for the two conclusions. It does not assert that one direction is more likely under \(H_0\), nor does it quantify the consequences of a true effect. An equal allocation is the special case \(\alpha_U=\alpha_L\). A one-sided design assigns a rejection budget to one direction only, but cannot provide the same prespecified two-direction stopping rule. In a multi-look design, all of these choices still require accounting for the sequential joint distribution.

A sound report gives the null model, the direction represented by each sign, the total target, the two directional targets, and the calibrated cumulative probabilities. It should also make clear whether the targets are exact equalities or conservative upper bounds. A numerical routine may return small rounding discrepancies, but the reported probabilities should be close enough to the targets to justify the stated precision.

A common mistake is to assign \(0.025\) to each tail of a \(0.05\) test and then spend \(0.025\) at every look in each direction. That would count the same directional budget repeatedly and ignore the dependence among looks. The correction is to define each target as the total probability of first crossing in that direction over the entire monitoring rule, and calibrate the boundary sequence to that cumulative probability.

Other warning signs include directional probabilities that do not sum to the reported overall error, limits whose signs do not match the stated benefit/harm convention, or calibration output based on paths that continued after an earlier efficacy crossing. If the stated null is not the point null used for calibration, or if the assumed information and covariance structure do not match the analysis, the advertised directional guarantees may not apply. These checks concern error accounting; meeting the error targets alone does not establish adequate performance under alternatives.