Stagewise Ordering of Group Sequential Outcomes

Why the Stopping Stage Is Part of the Outcome

In a group sequential test, the data are examined at planned looks, and the trial may stop before the final look. A stopped result therefore includes not only a test statistic but also the stage at which observation ended. Ranking results by the final-look statistic alone can discard information about when the trial crossed an efficacy boundary. Stagewise ordering makes that stopping stage an explicit part of the ranking.

This tutorial considers a one-sided test in which larger values of a standardized statistic are more adverse to $H_0$, and in which interim stopping is for efficacy. The ordering is a way to compare stopped outcomes; it is not, by itself, a p-value calculation. The introductory material on ordering stopped outcomes by evidence motivates why the stopping decision matters. Here the focus is the specific rule that uses stopping stage first and the statistic observed at that stage second.

Definition: Suppose a group sequential design has $J$ planned looks. Write $K$ for the stage at which the trial stops, and $Z_K$ for the one-sided standardized test statistic observed at that stage, oriented so that larger values indicate more evidence against $H_0$. A stopped outcome is represented for stagewise ordering by the pair $(K,Z_K)$, together with the stopping rule that determines which such pairs are attainable.

The Stagewise Ordering Rule

For the one-sided efficacy-stopping setting, stagewise ordering is lexicographic: compare stopping stages first, then compare statistics if the stages are the same. An outcome at an earlier stage is ranked as more extreme than every outcome at a later stage. Among outcomes at the same stage, the larger observed statistic is ranked as more extreme.

Definition: For two attainable stopped outcomes $(k,z)$ and $(k',z')$, say that $(k,z)$ is more extreme than $(k',z')$ under the stagewise ordering if $$ k<k' \quad\text{or}\quad (k=k' \text{ and } z>z'). $$ If $k=k'$ and $z=z'$, the two outcomes are tied under this ordering.

This rule is a deliberate convention about ordering outcomes, not an assertion that a smaller stage number always has a larger statistic. It gives priority to the fact that an efficacy boundary was crossed earlier. Within a fixed stage, it uses the direction of evidence specified for the one-sided test. The boundary and the rule that determines when the trial stops are needed to decide whether a proposed pair is actually possible.

For example, let the interim efficacy boundaries be $b_1,\ldots,b_{J-1}$, with stopping at the first interim stage $j$ for which $Z_j\ge b_j$, and completion at stage $J$ if no interim boundary is crossed. An outcome stopping at stage $k<J$ must have reached stage $k$ and crossed its boundary there. In particular, it cannot also have crossed an earlier efficacy boundary. A final-stage outcome occurs only after the trial has continued through the earlier looks. The ordering compares only outcomes that can arise under this rule.

A normal joint distribution for the sequential statistics is not required to state this ranking rule. For calibrated inference, however, the null distribution of the stopped outcome is essential: an ordering alone does not determine the probability of outcomes ranked at least as extreme as an observed result.

Worked Applications

Worked Example: Ranking Outcomes at Different Stages

Suppose a hypothetical three-look trial uses interim efficacy boundaries $b_1=2.80$ and $b_2=2.00$, and stops at the first boundary crossing. The final look is reached if neither interim boundary is crossed. Consider four possible stopped outcomes:

A: $(1,2.95)$; B: $(2,2.60)$; C: $(2,2.15)$; and D: $(3,2.40)$. For B and C to be attainable, suppose the stage 1 statistic was below 2.80. For D, suppose neither interim boundary was crossed.

First check attainability. A crosses the stage 1 boundary because $2.95\ge2.80$. B and C cross the stage 2 boundary because $2.60\ge2.00$ and $2.15\ge2.00$, respectively, and the stated stage 1 condition allows the trial to reach stage 2. D is a possible final outcome if its earlier statistics stayed below their boundaries.

Apply stage-first ranking: every stage 1 outcome outranks every stage 2 outcome, and every stage 2 outcome outranks every stage 3 outcome. Within stage 2, $2.60>2.15$, so B outranks C. The complete order from most to least extreme is A, B, C, D. Notice that D’s statistic, 2.40, is larger than C’s 2.15, but D is still ranked below C because it occurred later.

Worked Example: Comparing Two Outcomes at the Same Stage

In a hypothetical four-look trial, suppose two attainable stopped outcomes both occur at stage 3: E has $Z_3=2.31$, and F has $Z_3=2.08$. Assume the one-sided statistic is oriented so that larger values indicate more evidence against $H_0$.

The stages are equal, so the within-stage rule decides the ranking. Since $2.31>2.08$, E is more extreme than F. The arithmetic is simply $2.31-2.08=0.23$; this difference describes the separation of their standardized statistics, not a difference in treatment effect units.

The stagewise comparison does not say how probable either outcome is under $H_0$. That requires the joint null distribution and the stopping rule. It only specifies which outcome ranks farther in the one-sided ordering.

Worked Example: The Same Statistic at Different Stages

Consider a hypothetical three-look design with efficacy boundaries $b_1=2.70$ and $b_2=2.20$, followed by a final look. Outcome G stops at stage 1 with $Z_1=2.80$. Outcome H reaches the final look and has $Z_3=2.80$; suppose its stage 1 and stage 2 statistics were $1.40$ and $1.90$, so neither interim boundary was crossed.

Both reported statistics equal 2.80. But their stopping stages differ: G has $K=1$, while H has $K=3$. By definition, the earlier stage ranks first, so G is more extreme than H under stagewise ordering. H is attainable because $1.40<2.70$ and $1.90<2.20$, allowing the trial to reach the final look.

This example shows why the ordering is not equivalent to sorting all observations by the numerical value of $Z_K$. The same statistic can receive a different rank because the stage is also part of the ordered outcome.

Why the Rule Has This Form

A useful way to understand the definition is to list the outcomes that rank at least as extreme as a particular observed outcome $(k,z)$. Under the stated ordering, these comprise all attainable outcomes that stop before stage $k$, together with outcomes that stop at stage $k$ and have statistic at least $z$. Outcomes at later stages do not qualify, even if their statistics are numerically larger.

This description follows directly from the two clauses in the definition. If an outcome stops at stage $r<k$, it meets the earlier-stage clause. If it stops at stage $r=k$, it meets the within-stage clause exactly when its statistic is at least $z$. If it stops at $r>k$, it meets neither clause. Thus the stagewise ordering creates a clear boundary between outcomes ranked above and below the observed result, before any probabilities are assigned.

For example, if the observed outcome is $(2,2.30)$, the more-extreme-or-equal set consists of all attainable stage 1 efficacy stops and all attainable stage 2 outcomes with $Z_2\ge2.30$. It does not include stage 3 outcomes merely because their final statistics exceed 2.30. Determining the probability of this set under $H_0$ is a separate calculation.

Assumptions, Diagnostics, and Alternatives

The ordering is most straightforward when the design has a fixed schedule of looks, a prespecified stopping rule, a one-sided alternative, and a statistic whose larger values consistently mean stronger evidence against $H_0$. The stage-specific statistics should be defined on scales that make within-stage comparison meaningful. In common group sequential tests, standardized statistics such as $Z_j$ provide that scale. A statistic with reversed direction must be reoriented before applying the rule.

Before ranking outcomes, check the following:

Check the stopping rule. An interim outcome must satisfy the rule for stopping at that stage and must not have crossed an earlier stopping boundary.
Check the evidence direction. If smaller statistics support $H_a$, the within-stage inequality must be reversed or the statistic must be reoriented.
Check the outcome representation. The pair $(K,Z_K)$ captures the components used by this ordering, but the full observed path may be needed to evaluate probabilities or retain other information.
Check ties. Equal stage and equal statistic values tie under this definition. With a continuous statistic exact ties may have probability zero, but discrete statistics can make tie handling important.

Stagewise ordering is not the only possible way to order stopped outcomes. Ranking only by $Z_K$ ignores when stopping occurred; it can rank a later large statistic ahead of an earlier boundary crossing. Ranking by the maximum statistic seen during the trial is another possible rule, but it is not the same definition: it uses the largest observed value rather than prioritizing the stopping stage. These alternatives answer different ordering questions and can produce different sets of outcomes ranked at least as extreme as the observation.

The stagewise convention also has limitations. If a design stops early for futility as well as for efficacy, an early futility stop is not naturally interpreted as more evidence against $H_0$ merely because it is early. Applying the simple stage-first rule to every stopping cause could therefore conflict with the intended evidence direction. The stopping cause and the ordering convention must be specified; one should not silently treat every early stop as an efficacy stop. Likewise, a two-sided test needs a rule for handling direction and magnitude, which is not supplied by this one-sided definition.

A common mistake is to assume that the most extreme outcome must be the one with the largest observed statistic, regardless of stage. That reasoning substitutes a statistic-only ranking for the stagewise rule. The correction is to compare stages first and compare statistic values only when the stages match. A second mistake is to rank an impossible pair, such as an outcome declared to stop at stage 2 despite having crossed a stage 1 efficacy boundary. Always verify attainability under the stopping rule before interpreting an ordering.

Key takeaway: For the one-sided efficacy-stopping case, stagewise ordering ranks an earlier stopping stage as more extreme; within the same stage, it ranks the larger statistic as more extreme. The rule defines a ranking, while null probabilities and p-value validity require additional work.

Check Your Understanding

Answer each question using the one-sided stagewise ordering defined above. Brief solutions follow the questions.

Which outcome is more extreme: $(2,2.40)$ or $(3,3.10)$?
At the same stage, how do $(3,1.95)$ and $(3,2.05)$ rank?
With a stage 1 efficacy boundary of 2.50, can a trial stop at stage 2 if $Z_1=2.70$? Explain.
For an observed outcome $(2,2.20)$, which stages and within-stage statistics can be at least as extreme under this ordering?
Why should the simple stage-first rule not automatically be applied to early futility stops?

Solutions. (1) $(2,2.40)$ is more extreme because stage 2 precedes stage 3, even though 3.10 is larger than 2.40. (2) $(3,2.05)$ is more extreme because the stages match and $2.05>1.95$. (3) No. Since $2.70\ge2.50$, the stage 1 boundary was crossed, so the trial should have stopped at stage 1. (4) All attainable stage 1 outcomes, plus attainable stage 2 outcomes with statistic at least 2.20; no later-stage outcomes qualify. (5) A futility stop generally reflects insufficient evidence for efficacy, so treating it as more extreme solely because it happened earlier can contradict the one-sided evidence direction.