Why Per-Tail Alpha Does Not Ignore Cross-Look Dependence

Why the Sum of Per-Look Probabilities Is Not the Overall Error

A group sequential test can reject $H_0$ at more than one planned analysis. In a two-sided design, rejection can also occur in either direction. The overall type I error is therefore about the chance of at least one false rejection during the monitoring process—not the sum of the chances attached separately to each look and tail.

Adding those separate probabilities is still useful: it gives a conservative upper bound. But it generally does not give the exact overall type I error, because the same trial outcome can cross a boundary at more than one look. The overlap between those crossing events depends on the joint distribution of the test statistics.

Definition: Let $A_i^+$ be the event that the test statistic crosses the upper efficacy boundary at look $i$ under $H_0$, and let $A_i^-$ be the event that it crosses the lower efficacy boundary. For $K$ looks, the overall type I error is $$ \alpha_{\mathrm{overall}}=P_0\left(\bigcup_{i=1}^{K}(A_i^+\cup A_i^-)\right), $$ where $P_0$ denotes probability under the null hypothesis. A per-look, per-tail probability such as $P_0(A_i^+)$ is a marginal probability; it is not, by itself, an increment of overall type I error.

This definition uses the event of crossing at least one boundary. It agrees with the probability of a false rejection under a stopping rule that stops at the first efficacy crossing: later crossings cannot add a second rejection after the trial has stopped. Equivalently, one can define disjoint events according to the first look and direction of rejection, then add their probabilities.

The Union Bound and Its Limitation

For any finite set of events, the probability of their union is no larger than the sum of their probabilities. Applying this union bound to all upper- and lower-tail crossings gives

\alpha_{\mathrm{overall}} \leq \sum_{i=1}^{K}\left\{P_0(A_i^+)+P_0(A_i^-)\right\}.

The inequality does not require independence. It is valid whether the test statistics at different looks are independent, positively correlated, or related in some other way. But the sum counts an outcome again each time it belongs to another crossing event. The union counts that outcome only once. Thus, when events overlap, the sum is larger than the union probability.

The exact two-event identity makes the issue visible:

P_0(A\cup B)=P_0(A)+P_0(B)-P_0(A\cap B).

With many events, inclusion-exclusion adds pairwise overlaps, subtracts triple overlaps, and continues through higher-order intersections. The probability of an overlap is determined by the joint null distribution, not by the marginal tail probabilities alone. In particular, knowing every $P_0(A_i^+)$ and $P_0(A_i^-)$ does not usually identify $\alpha_{\mathrm{overall}}$.

In the usual canonical group sequential model under $H_0$, the standardized statistics $Z_i$ at information times $t_1<\cdots<t_K$ have a joint normal distribution with mean zero, unit variances, and

\operatorname{Corr}(Z_i,Z_j)=\sqrt{\frac{t_i}{t_j}},\qquad i<j.

This correlation arises when the accumulated score has independent increments and is standardized by the square root of its information. It means the statistics at successive looks are generally dependent. The boundaries and this joint distribution together determine the crossing probabilities. The marginal probability at an individual look is not enough.

Worked Examples

Worked Example: Two Upper-Tail Crossings

Consider a hypothetical two-look design under $H_0$. Suppose a joint null calculation gives $P_0(A_1^+)=0.020$, $P_0(A_2^+)=0.030$, and $P_0(A_1^+\cap A_2^+)=0.012$. Find the probability of an upper-boundary crossing at either look.

Using the two-event identity,

P_0(A_1^+\cup A_2^+) =0.020+0.030-0.012 =0.038.

The sum of the marginal probabilities is $0.050$, which is a valid upper bound, but it overstates the union probability by $0.012$. The overlap represents null outcomes counted at both looks in the marginal sum. These hypothetical probabilities illustrate the calculation; they are not claims about a particular study or a particular boundary design.

Worked Example: Two Looks and Two Tails

Suppose a hypothetical two-sided design has four crossing events, $A_1^+, A_1^-, A_2^+, A_2^-$, each with marginal null probability $0.0125$. At a given look, upper and lower crossings are mutually exclusive. Assume a joint null calculation gives a total of $0.008$ for the sum of the four pairwise intersection probabilities across looks, and that no three crossing events can occur together. Calculate the overall type I error.

The four marginal probabilities sum to

$$ 4(0.0125)=0.050. $$

Because same-look opposite-tail events are disjoint and triple intersections are zero under the stated assumptions, inclusion-exclusion gives

\alpha_{\mathrm{overall}} =0.050-0.008 =0.042.

Thus $0.050$ is the union-bound value, whereas $0.042$ is the exact overall error for the hypothetical joint probabilities specified here. The four marginal probabilities alone would not reveal the $0.008$ overlap total, so they would not determine the exact answer.

Worked Example: Same Marginal Probabilities, Different Overall Error

Let $A$ and $B$ be null crossing events at two looks, with $P_0(A)=0.020$ and $P_0(B)=0.030$. First suppose $A$ is contained in $B$, so $P_0(A\cap B)=0.020$. Then

P_0(A\cup B)=0.020+0.030-0.020=0.030.

Now suppose instead that the events are disjoint, so $P_0(A\cap B)=0$. Then

P_0(A\cup B)=0.020+0.030-0=0.050.

The marginal probabilities have not changed, but the union probability has. These two dependence patterns illustrate the range of possibilities; they are not a claim that either pattern is the joint distribution of a particular canonical group sequential design. Under independence, for comparison, the intersection would be $0.020(0.030)=0.0006$, giving a union probability of $0.0494$. Independence is not a default assumption for repeated group sequential statistics.

Marginal Probabilities, First Crossings, and Calibration

The union-bound calculation is a safe check, not usually the target of design calibration. If there are $K$ looks and two tails, and every marginal tail probability is $q$, then

\alpha_{\mathrm{overall}}\leq 2Kq.

Choosing $q\leq\alpha/(2K)$ therefore guarantees an overall error no greater than $\alpha$, regardless of dependence. This Bonferroni allocation can be conservative because it ignores overlap. It does not show that the overall error equals $2Kq$, nor that a less conservative design has been calibrated to the desired level.

An exact group sequential calculation instead uses the joint null distribution and the full stopping rule. One convenient representation partitions the overall rejection event into disjoint first-crossing events. Let $F_i^+$ and $F_i^-$ denote rejection for the first time at look $i$ in the upper or lower tail, respectively. Then

\alpha_{\mathrm{overall}} =\sum_{i=1}^{K}\left\{P_0(F_i^+)+P_0(F_i^-)\right\}.

These first-crossing events are disjoint by definition, so their probabilities add exactly. Their probabilities are not generally equal to the marginal probabilities $P_0(A_i^+)$ and $P_0(A_i^-)$: a crossing at look $i$ is a first crossing only if the trial did not already cross a boundary at an earlier look.

This also clarifies the difference between a marginal per-look tail probability and an error-spending increment. A marginal probability asks whether the statistic at a specified look exceeds its boundary under $H_0$, without removing paths that would already have stopped. A sequential increment is calibrated within the monitoring rule so that the accumulated probability of rejection is controlled as planned. Treating these quantities as interchangeable is a common source of error.

Diagnostics, Alternatives, and Limitations

When reviewing a claimed overall type I error, check what each reported probability represents. Is it a marginal tail probability at a look, a probability of first crossing there, or cumulative error through that look? These are distinct quantities. Also check that the calculation includes both tails when the design is two-sided and respects the actual stopping rule.

The union bound is appropriate when a conservative guarantee is sufficient or when joint probabilities are unavailable. Exact joint-distribution calculations are preferable when a design needs a specified overall error rate without unnecessary conservatism. The introductory material on joint null error across two tails establishes why both directions belong in the rejection event; the additional issue here is that crossings at different looks overlap as well. Neither marginal tail allocation nor the union bound alone resolves that cross-look dependence.

In standard group sequential calculations, joint-normal integration or recursion can account for the dependence among looks and evaluate the first-crossing probabilities. Such calculations require the assumed joint model and information times to represent the design adequately. If the monitoring schedule, statistic, or data-generating assumptions differ from those used in calibration, the computed error rate may no longer apply. A sum of nominal per-look probabilities cannot diagnose or repair that mismatch.

Pitfall: “Each of four tail-look probabilities is $0.0125$, so the overall error is exactly $0.050$.”
Correction: The sum is $0.050$, which gives a union bound. The exact error is the probability of the union and depends on overlaps among the crossing events.
Diagnostic: If a calculation adds marginal tail probabilities and reports equality without a disjointness argument or a joint-probability calculation, it has not established the exact overall error.

Key takeaway: Overall type I error is one probability: the null probability of at least one rejection across all looks and both tails. Adding marginal per-look tail probabilities gives a dependence-free upper bound, not generally that exact probability.

Check Your Understanding

Answer each question using the distinction between marginal crossing probabilities and the probability of their union.

Two null crossing events have probabilities $0.01$ and $0.02$, and their intersection has probability $0.004$. What is the probability of at least one crossing?
For six looks in a two-sided design, each of the twelve marginal tail probabilities is $0.004$. What upper bound does the union bound give for overall type I error?
Why can marginal probabilities at every look be insufficient to calculate exact overall type I error?
What makes first-crossing events different from marginal look-specific crossing events?

Solutions. (1) $0.01+0.02-0.004=0.026$. (2) The union bound is $12(0.004)=0.048$, so the overall error is at most $0.048$, not necessarily equal to it. (3) The marginal probabilities do not specify intersections among crossing events; those intersections depend on their joint null distribution. (4) A first-crossing event requires no earlier crossing, so first-crossing events are disjoint and add exactly to the overall rejection probability, while marginal events do not exclude earlier crossings.