Worked Comparison Of Boundary Decisions

From Boundary Tables to Stopping Decisions

A classical group sequential design specifies interim analyses and a boundary at each one. The practical question is not merely whether an observed statistic looks large. It is whether the statistic crosses the boundary at the current look, and, if so, whether that is the first crossing. This tutorial applies that rule to the same observed sequences under three designs: Pocock, O’Brien-Fleming, and a specified Wang-Tsiatis design.

The comparison is about decisions for a given sequence of statistics, not about choosing a design or calculating its operating characteristics. The shape and interpretation of these boundaries are developed in the earlier material on classical boundary shapes and geometries; the earlier material on nominal p-values also explains why a look-specific nominal p-value is not, by itself, the overall significance level.

Setup and assumptions: Consider a two-sided efficacy design with \(K\) planned analyses, information fractions \(0<t_1<\cdots<t_K=1\), and observed standardized statistics \(Z_1,\ldots,Z_K\). Assume the statistics are computed for the same treatment contrast and outcome, with larger positive values favoring treatment and larger negative values favoring control. At look \(i\), let \(u_i\) be the positive critical value; the symmetric boundaries are \(-u_i\) and \(u_i\). The design and analysis schedule are prespecified, and the supplied critical values are calibrated for the intended overall type I error under the design’s null model. The decision rule is to stop for efficacy at the first look \(i\) for which \(|Z_i|\geq u_i\). If no boundary is crossed, continue to the next planned look; if none is crossed by \(K\), there is no efficacy stop under this rule.

This setup presumes that the standardized statistics and their joint distribution are appropriate for the chosen group sequential design. In the usual canonical approximation under \(H_0\), the statistics are jointly normal with mean zero and covariance \( \operatorname{Cov}(Z_i,Z_j)=\sqrt{t_i/t_j} \) for \(i\leq j\). The repeated statistics are therefore dependent, not independent. In practice, the design’s stated boundary table is the decision reference; a sequence of observed values alone does not establish that the boundary was correctly calibrated.

For a one-sided efficacy design, the same first-crossing logic applies to the upper boundary alone: stop at the first \(i\) with \(Z_i\geq u_i\). Here the examples use symmetric two-sided boundaries so that a crossing in either direction is visible. The sign indicates the direction of the estimated effect; it does not change whether the absolute statistic has crossed.

The Three Boundary Patterns

Pocock boundaries use a constant critical value across looks. O’Brien-Fleming boundaries are much more stringent early and relax toward the end. A Wang-Tsiatis boundary uses a shape parameter to control how quickly the cutoff declines. For this worked comparison, take four equally spaced information fractions and the following hypothetical, prespecified design output. The values are rounded for presentation; treat the table as the supplied calibrated design rather than recalibrating it from the rounded entries.

Look \(i\)	Information fraction \(t_i\)	Pocock \(u_i\)	O’Brien-Fleming \(u_i\)	Wang-Tsiatis, \(\delta=0.25\), \(u_i\)
1	0.25	2.36	3.47	2.83
2	0.50	2.36	2.45	2.38
3	0.75	2.36	2.00	2.15
4	1.00	2.36	1.74	2.00

For the Wang-Tsiatis column, the shape is represented by \(u_i=c/t_i^{1/2-\delta}\), with \(\delta=0.25\) and a design-specific constant \(c\). The table displays rounded values from the specified design. The formula describes the shape; the calibrated constant and resulting cutoffs determine the actual decision. Neither a shape formula alone nor a rounded table should be treated as a substitute for the complete design specification.

Decision rule: For each design, scan the observed statistics in planned look order. At each look compare \(|Z_i|\) with that design’s \(u_i\). The first look satisfying \(|Z_i|\geq u_i\) is the efficacy stopping look. Once it is reached, later statistics do not alter the decision for that sequential procedure.

Worked Comparisons

Worked Example: Different First-Crossing Looks

Suppose a hypothetical trial has observed standardized statistics \(Z_1=2.20\), \(Z_2=2.42\), \(Z_3=2.10\), and \(Z_4=2.05\). Apply the table above separately to each design.

Pocock. At look 1, \( |Z_1|=2.20<2.36 \), so continue. At look 2, \( |Z_2|=2.42\geq2.36 \). The first crossing is look 2, so the Pocock rule stops for efficacy there.

O’Brien-Fleming. At look 1, \(2.20<3.47\); at look 2, \(2.42<2.45\). At look 3, \(2.10\geq2.00\), so the first crossing is look 3. The statistic at look 4 is not used to revise this stopping decision.

Wang-Tsiatis with \(\delta=0.25\). At look 1, \(2.20<2.83\). At look 2, \(2.42\geq2.38\), so this design stops at look 2. Thus the same observed path leads to stopping at look 2 under Pocock and the stated Wang-Tsiatis design, but at look 3 under O’Brien-Fleming.

All statistics in this example are positive, so each crossing favors treatment under the stated sign convention. The comparison does not show that one result is more statistically valid than another: each decision follows its own prespecified boundary.

Worked Example: A Crossing in the Negative Direction

Now suppose a separate hypothetical sequence is \(Z_1=-1.00\), \(Z_2=-2.50\), \(Z_3=-2.20\), and \(Z_4=-2.10\). Use the same two-sided boundary table.

Pocock. At look 1, \( |-1.00|=1.00<2.36 \). At look 2, \( |-2.50|=2.50\geq2.36 \); stop at look 2.

O’Brien-Fleming. At look 1, \(1.00<3.47\), and at look 2, \(2.50\geq2.45\). Stop at look 2.

Wang-Tsiatis. At look 1, \(1.00<2.83\). At look 2, \(2.50\geq2.38\); stop at look 2.

Each rule registers a boundary crossing because each uses \(|Z_i|\). The negative sign means the observed estimate is in the direction favoring control, not treatment. In a real study, whether that direction supports a prespecified efficacy claim depends on the estimand, hypotheses, and design; a two-sided crossing should not be relabeled as evidence of benefit in the opposite direction.

Worked Example: No Efficacy Boundary Is Crossed

For a third hypothetical sequence, let \(Z_1=1.50\), \(Z_2=1.80\), \(Z_3=1.90\), and \(Z_4=1.70\). Determine whether each design stops for efficacy.

Pocock. The absolute statistics \(1.50, 1.80, 1.90, 1.70\) are all less than the constant cutoff \(2.36\). There is no crossing.

O’Brien-Fleming. Compare in order: \(1.50<3.47\), \(1.80<2.45\), \(1.90<2.00\), and \(1.70<1.74\). No crossing occurs, including at the final look.

Wang-Tsiatis. The comparisons are \(1.50<2.83\), \(1.80<2.38\), \(1.90<2.15\), and \(1.70<2.00\). Again, there is no efficacy crossing.

The correct conclusion is that none of these efficacy rules calls for stopping early, and none crosses its boundary at the maximum planned look. This is not the same as proving \(H_0\), and it does not by itself determine what other prespecified trial rules, such as a futility rule, would require.

Why the First-Crossing Rule Is Essential

A group sequential procedure is defined by its full pathwise stopping rule, not by treating every look as a fresh, independent test. If \(A_i\) denotes the event that the statistic first crosses at look \(i\), then \(A_i=\{|Z_1|<u_1,\ldots,|Z_{i-1}|<u_{i-1},|Z_i|\geq u_i\}\). The events \(A_i\) are disjoint: a trial cannot have its first crossing at two different looks. Consequently, the probability of stopping for efficacy by look \(K\) is the sum of the probabilities of these first-crossing events, not a calculation that assumes the looks are independent.

P(\text{cross by look }K\mid H_0) =\sum_{i=1}^{K}P(A_i\mid H_0).

This gives a short justification for why the decision must be read sequentially. If look \(i\) is the first crossing, then all earlier comparisons failed and the stopping rule is met at \(i\). Later observations belong to a path the procedure would not reach after stopping. Using those later values to change the recorded decision would define a different procedure, with different error properties.

Fix the design.
Record the planned looks, information fractions, sidedness, and the correct boundary column, including any Wang-Tsiatis shape parameter.

Check one look at a time.
For a symmetric two-sided design compare \(|Z_i|\) with \(u_i\); for an upper one-sided design compare \(Z_i\) with its upper cutoff.

Stop at the first crossing.
State the look and direction. If there is no crossing, continue through the planned schedule and report that no efficacy boundary was crossed.

Diagnostics, Alternatives, and Limitations

A practical check is to write the comparison explicitly at every look up to the first crossing. This catches errors caused by comparing with the wrong design column, overlooking a cutoff that changes over time, or skipping an earlier look. Verify that the observed statistics belong to the planned analysis schedule and use the same standardization assumed by the boundary design. If the analysis timing, test statistic, variance model, or sidedness differs from the specification, the supplied cutoffs may no longer give the intended operating characteristics.

Do not infer a design’s overall type I error simply by inspecting one observed statistic or its nominal p-value. The overall error depends on the chance of crossing at any planned look under the null and on the joint distribution across looks. Similarly, rounded critical values can be enough for an illustrative decision but may be inadequate when an observed statistic lies very close to a cutoff. Operational decisions should use the prespecified full-precision values and analysis plan.

The main alternative in this comparison is not a different way to read the same boundary, but a different prespecified boundary design. Pocock has a constant cutoff; O’Brien-Fleming spends relatively little opportunity for crossing early and has a lower final cutoff; Wang-Tsiatis provides intermediate shapes depending on \(\delta\). For a fixed sequence, these patterns can yield different stopping looks, as the first example showed. That comparison alone does not establish which design is preferable: overall error control and other design priorities must be considered separately.

These examples address efficacy boundaries only. They do not add futility rules, modify the analysis schedule, estimate expected sample size, or choose a maximum information level. Nor do they justify unplanned repeated looks: adding analyses changes the stopping procedure and may change its error properties. Apply the boundary table only under the conditions for which the design was specified.

Common pitfall: “The statistic crossed the boundary at a later look, so I can report that later result even though the trial already crossed earlier.” This is wrong because a sequential rule stops at the first crossing. The correction is to scan in order, record the first crossing as the decision, and treat subsequent values as unavailable for that rule’s decision.

Check Your Understanding

Use the four-look table in this tutorial. Give the stopping look, or state that no efficacy boundary is crossed.

For \(Z=(2.30,2.40,1.90,1.80)\), what is the first crossing under Pocock?
For \(Z=(3.40,2.30,2.05,1.80)\), when does the O’Brien-Fleming design first cross?
For \(Z=(1.00,2.30,2.20,2.05)\), when does the stated Wang-Tsiatis design first cross?
Why can a statistic observed after the first crossing not be used to move the stopping decision to a later look?

Solutions. (1) Look 1 has \(2.30<2.36\); look 2 has \(2.40\geq2.36\), so Pocock stops at look 2. (2) At looks 1 and 2, \(3.40<3.47\) and \(2.30<2.45\); at look 3, \(2.05\geq2.00\), so O’Brien-Fleming stops at look 3. (3) The first two comparisons are \(1.00<2.83\) and \(2.30<2.38\); at look 3, \(2.20\geq2.15\), so Wang-Tsiatis stops at look 3. (4) The rule is defined by the first crossing; after that crossing, later data do not change the decision for the specified sequential procedure.