Decomposing Stagewise Tail Probabilities by Look

Why Decompose a Stagewise Tail Probability?

A stagewise p-value is a null probability over stopped outcomes that are at least as extreme as the outcome observed, according to a specified ordering. That set of outcomes can contain trials stopping at several different looks. Its probability can therefore be calculated by adding the probability contributions from each stopping look.

This decomposition is useful for understanding where a p-value comes from and for checking a numerical calculation. In particular, it distinguishes the probability of qualifying outcomes that stop at a given look from the probability conditional on stopping at that look. Those are generally different quantities.

Stopping Events and the Look-Specific Tail

Suppose there are \(K\) planned looks. Write \(T\) for the look at which the trial stops, so \(T\) takes one value in \(\{1,\ldots,K\}\). Let \(Z_j\) denote the test statistic at look \(j\), and let \(R_j\) be the region of possible statistic histories that makes the trial stop at look \(j\). For example, \(R_j\) includes histories meeting a stopping boundary at look \(j\) while satisfying the required conditions for not having stopped earlier. At the final look, every history not stopped earlier must stop.

The method requires a null distribution for the statistics and a prespecified ordering of stopped outcomes by evidence. Denote the observed stopped outcome by \(o_{\mathrm{obs}}\), and write \(o\succeq o_{\mathrm{obs}}\) when outcome \(o\) is at least as extreme under that ordering. The ordering may depend on both the stopping look and the statistic observed there; it must be applied consistently across the whole stopped-outcome space.

Definition: The stagewise p-value is the null probability of the set of stopped outcomes ordered at least as extreme as the observed outcome. The contribution from look \(j\) is the null probability that the trial stops at \(j\) and its stopped outcome is in that tail set. These contributions are joint probabilities; they are not conditioned on \(T=j\).

Under the canonical group sequential null model, the vector of statistics has a multivariate normal distribution. With information levels \(I_1<\cdots<I_K\), one common formulation is \(Z_j\sim N(0,1)\) under \(H_0\), with \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{I_i/I_j}\) for \(i<j\). This formulation assumes the usual independent-increment information model and a fixed look schedule. The decomposition itself only needs a specified joint null distribution and a well-defined stopping rule; other valid models can be used when appropriate.

For an observed outcome at look \(k\), let \(A_j(o_{\mathrm{obs}})\) be the set of statistic histories that both stop at look \(j\) and are ordered at least as extreme as \(o_{\mathrm{obs}}\). If \(f_0\) is the joint null density of the full statistic vector, then

p_{\mathrm{stage}}(o_{\mathrm{obs}}) = \sum_{j=1}^{K} P_0\{(Z_1,\ldots,Z_j)\in A_j(o_{\mathrm{obs}})\} = \sum_{j=1}^{K}\int_{A_j(o_{\mathrm{obs}})} f_0(z_1,\ldots,z_K)\,d z_1\cdots d z_K.

Here the integral may be over the full vector even though the stopped outcome is determined by the history through look \(j\); later coordinates can be integrated out. Splitting the sum around the observed look gives the central decomposition:

p_{\mathrm{stage}} = \underbrace{\sum_{j<k}P_0(A_j)}_{\text{stops earlier}} + \underbrace{P_0(A_k)}_{\text{stops at the observed look}} + \underbrace{\sum_{j>k}P_0(A_j)}_{\text{stops later}}.

The sets \(A_1,\ldots,A_K\) are disjoint because a trial stops at exactly one look. Thus their probabilities add without double-counting. This is the entire justification for the decomposition: the tail event is partitioned according to the unique value of \(T\).

Worked Examples of the Decomposition

Worked Example: Reading Contributions Around the Observed Look

Consider a hypothetical three-look trial. The observed outcome stops at look \(k=2\). A prespecified stagewise ordering and a null integration have identified the qualifying regions \(A_1,A_2,A_3\). Suppose the computed joint null probabilities are \(P_0(A_1)=0.0042\), \(P_0(A_2)=0.0187\), and \(P_0(A_3)=0.0026\).

There is one earlier look, the observed look itself, and one later look. Substitution into the decomposition gives

p_{\mathrm{stage}} = 0.0042+0.0187+0.0026 = 0.0255.

The earlier-stop contribution is 0.0042, the contribution from look 2 is 0.0187, and the later-stop contribution is 0.0026. The stagewise p-value is 0.0255. In this hypothetical calculation, qualifying outcomes stopping at look 1 or look 3 are part of the tail even though the observed trial stopped at look 2.

Each number is a probability under \(H_0\) of both stopping at that look and satisfying the ordering criterion. For example, 0.0187 is not the probability of qualifying given that the trial stopped at look 2.

Worked Example: Summing a Discrete Null Distribution

To make the event partition explicit, consider a hypothetical finite null distribution for stopped outcomes. An ordering has been fixed, and the observed outcome is at look 2. A null enumeration groups outcomes into whether they qualify for the stagewise tail. The total null probability at each look and the qualifying part are shown below.

Stopping look	Total probability of stopping there	Qualifying tail probability
1	0.20	0.006
2	0.30	0.021
3	0.50	0.004

The total probabilities across stopping looks sum to \(0.20+0.30+0.50=1.00\), as required. The qualifying probabilities are subsets of those totals: at each look, the qualifying mass is no greater than the total mass. Since the observed outcome is at look 2, the tail includes the qualifying mass from all three rows:

p_{\mathrm{stage}} = P_0(A_1)+P_0(A_2)+P_0(A_3) = 0.006+0.021+0.004 = 0.031.

Thus the earlier, same-look, and later contributions are 0.006, 0.021, and 0.004. The remaining null probability, \(1-0.031=0.969\), is assigned to stopped outcomes outside the stagewise tail. This finite example illustrates the partition; in a continuous group sequential model the corresponding probabilities are usually obtained by integrating over regions of the joint null distribution.

Worked Example: Joint Contribution Versus Conditional Probability

Use the same hypothetical null distribution. The probability of stopping at look 2 is 0.30, and the qualifying tail probability that stops there is 0.021. The look-2 contribution to the stagewise p-value is the joint probability 0.021. If the question instead asks for the probability of qualifying given that the trial stopped at look 2, the calculation is

P_0(A_2\mid T=2) = \frac{P_0(A_2)}{P_0(T=2)} = \frac{0.021}{0.30} = 0.070.

The conditional probability is 0.070, while the look-2 contribution to the stagewise p-value is 0.021. The stagewise p-value includes the joint qualifying mass from all looks, \(0.006+0.021+0.004=0.031\); it is not the conditional probability given the observed stopping look.

Why the Decomposition Holds, and How to Check It

Let \(E\) be the event that a stopped outcome is at least as extreme as the observed outcome. Because the stopping rule assigns exactly one stopping look, the events \(E\cap\{T=j\}\) are mutually exclusive and their union is \(E\). Additivity of probability therefore gives

P_0(E) = P_0\left(\bigcup_{j=1}^{K}(E\cap\{T=j\})\right) = \sum_{j=1}^{K}P_0(E\cap\{T=j\}).

Each term is the look-specific contribution. Grouping the terms with \(j<k\), \(j=k\), and \(j>k\) yields the earlier, same-look, and later parts. No assumption that the looks are independent is used in this probability identity. Dependence does matter when calculating the individual terms; for example, canonical group sequential statistics are correlated across looks.

A careful calculation checks both the event definition and the arithmetic. The stopping region \(R_j\) must exclude histories that already stopped, and the ordering must be applied to the stopped outcome rather than to a statistic from an unobserved future look. For a continuous statistic, boundary equalities usually have probability zero under a continuous null distribution. For discrete outcomes or conventions that assign ties to the tail, the tie rule must be specified and used consistently.

Check the stopping partition. Under the null model, the probabilities \(P_0(T=j)\) should sum to 1, up to numerical error.
Check each contribution. Each \(P_0(A_j)\) must be between 0 and \(P_0(T=j)\).
Check the total. The sum must be between 0 and 1, and it cannot exceed the total probability of stopping across the looks.
Check the ordering region. A qualifying outcome must satisfy both the stopping rule for its look and the chosen evidence ordering relative to the observed outcome.
Check numerical accuracy. Integration error, missed regions, or probability-mass leakage can distort individual contributions even when the final sum looks plausible. Compare the total with an independent calculation or a simulation check when available.

The decomposition also clarifies what is and is not being conditioned on. A stagewise p-value uses the null distribution over all stopped outcomes. Restricting attention to \(T=k\) and dividing by \(P_0(T=k)\) changes the probability statement to one conditional on the stopping look. That may answer a different inferential question, but it is not the same calculation.

Common Pitfall and Limitations

Common mistake: “The observed trial stopped at look \(k\), so only the look-\(k\) tail probability belongs in the stagewise p-value.” This omits qualifying outcomes that would have stopped before or after \(k\). The correction is to define the full ordered tail event over stopped outcomes and sum its joint null probability across every look.

This decomposition does not choose the ordering, establish that an ordering is valid, or determine the null model. Those must be specified separately. Different orderings can produce different stagewise p-values for the same stopped data, and the look-specific contributions may be difficult to calculate when boundaries and joint distributions are complex. The decomposition remains useful in those settings because it identifies exactly which probabilities need to be calculated and how they combine.

Check Your Understanding

Answer each question, then compare with the brief solutions.

An observed outcome is at look 3 in a four-look design. Which stopping-look contributions belong in the stagewise p-value?
Suppose \(P_0(A_1)=0.003\), \(P_0(A_2)=0.012\), and \(P_0(A_3)=0.005\), with no qualifying outcomes at later looks. Calculate the stagewise p-value.
If \(P_0(A_2)=0.016\) and \(P_0(T=2)=0.40\), calculate the qualifying probability conditional on stopping at look 2. Is it the look-2 contribution to the stagewise p-value?
Why can contributions from looks after the observed look be nonzero?

Solutions. 1. Include all qualifying contributions from looks 1, 2, 3, and 4, grouped as earlier (1 and 2), same look (3), and later (4). 2. Add the joint probabilities: \(0.003+0.012+0.005=0.020\). 3. The conditional probability is \(0.016/0.40=0.040\); the look-2 contribution is the joint probability 0.016, so they differ. 4. Under the null, other trials may continue beyond look 3 and later stop with outcomes that meet the prespecified evidence ordering relative to the observed outcome.