Unequal-Look O'Brien-Fleming Example

Why Unequal Look Spacing Matters

A group sequential design can schedule analyses at information fractions that are not equally spaced. For example, a trial might plan looks after 15%, 60%, and 100% of its maximum information has accrued. The O’Brien–Fleming boundary still has its characteristic high early threshold and lower later threshold, but each cutoff depends on the information fraction at that particular look. The decisions therefore require comparing each observed statistic with its own prespecified critical value.

This tutorial considers a one-sided efficacy test in a prespecified direction. A positive Z statistic is evidence in that direction. The purpose is to calculate and interpret the boundary values and identify crossings—not to redesign the schedule after seeing results.

Definition: Let $t_k=I_k/I_{\max}$ be the information fraction at look $k$, where $I_k$ is the information available at that look and $I_{\max}$ is the planned maximum information. An O’Brien–Fleming efficacy boundary has critical value $$ b_k=\frac{c}{\sqrt{t_k}}, $$ where $c$ is a positive constant calibrated for the design’s planned looks, sidedness, and overall type I error rate. At look $k$, efficacy is declared if the observed statistic $Z_k$ is at least $b_k$.

Model, Assumptions, and Calibration

The standard canonical model treats the sequential Z statistics under $H_0$ as jointly normal with mean zero, unit variance, and correlations $$ \operatorname{Corr}(Z_i,Z_j)=\sqrt{\frac{t_i}{t_j}}, \qquad t_i\leq t_j. $$ This model represents accumulating information: statistics at later looks contain the earlier data, so the looks are not independent. Increments of the underlying score process are independent under the null in the canonical model.

Using a boundary procedure requires planned information fractions in increasing order, a valid Z-statistic approximation at each look, and consistent definitions of the endpoint, analysis population, and test direction across looks. The information fractions must describe the planned information accumulation; they are not automatically the fractions of calendar time or participants enrolled. The schedule and boundary should be fixed before examining interim results. If the test is two-sided, the design must instead specify lower and upper boundaries and calibrate the overall error allocation accordingly.

The constant $c$ is not chosen by simply applying the ordinary single-look critical value at every analysis. It is calibrated so that the probability of crossing any planned efficacy boundary under $H_0$ equals the design’s target type I error rate. Under the canonical model, calibration uses the joint distribution of the planned Z statistics and solves $$ P_{H_0}\left(\bigcup_{k=1}^{K}\left\{Z_k\geq \frac{c}{\sqrt{t_k}}\right\}\right)=\alpha. $$ The exact value depends on the schedule, the number of looks, and the sidedness. For a practical analysis, use the calibrated value or critical values specified for that design; do not infer $c$ from a single cutoff rounded in a table.

Why the Cutoff Scales With Information

The O’Brien–Fleming shape can be understood using the unstandardized evidence in the canonical model. A standardized statistic at information fraction $t$ scales roughly as the accumulated score divided by its standard deviation, which grows in proportion to $\sqrt{t}$. Thus the corresponding score scale is proportional to $\sqrt{t}Z$. Holding the evidence threshold on that scale at a constant $c$ gives the boundary condition $\sqrt{t}Z\geq c$.

\sqrt{t}Z\geq c \quad\Longleftrightarrow\quad Z\geq \frac{c}{\sqrt{t}}.

This is the inverse-square-root shape: with less information, a more extreme standardized statistic is required. The derivation explains the shape, while calibration determines the constant appropriate to the full sequence of looks.

Calculating Cutoffs at Unequal Looks

Consider three prespecified information fractions $t_1=0.15$, $t_2=0.60$, and $t_3=1.00$. Suppose the design’s calibration for its specified error rate and sidedness supplies $c=1.98$, rounded to three significant figures. The critical values are obtained separately at each fraction:

b_1=\frac{1.98}{\sqrt{0.15}}=5.112,\qquad b_2=\frac{1.98}{\sqrt{0.60}}=2.556,\qquad b_3=\frac{1.98}{\sqrt{1.00}}=1.980.

These cutoffs are not equally spaced, because the information fractions are not equally spaced. The first look occurs with relatively little information, so its threshold is very high. The second look has accumulated substantially more information and has a lower threshold. At the final look, the cutoff is $c$ itself because $t_3=1$. Values shown here are illustrative of the calculation; in an actual design, the calibrated constant and any reported cutoffs should come from that design’s calibration.

Worked Example: No Crossing Until the Final Look

Use the three-look design above. In a hypothetical trial, the observed statistics are $Z_1=2.10$, $Z_2=2.50$, and $Z_3=2.05$, assuming the study continues whenever the efficacy boundary has not been crossed. Determine the efficacy decision at each look.

At look 1, the cutoff is $5.112$. Since $2.10<5.112$, the statistic does not cross the boundary.

At look 2, the cutoff is $2.556$. Since $2.50<2.556$, there is no crossing at this look either. Although the statistic is close to the cutoff, being close is not the same as crossing it.

At look 3, the cutoff is $1.980$. Since $2.05\geq1.980$, the statistic crosses the efficacy boundary. The design’s efficacy stopping rule is met at the final analysis. This conclusion concerns the prespecified sequential rule; it does not mean that the evidence at earlier looks could be compared with the final cutoff.

Reading a Path of Interim Results

In a group sequential design, a boundary crossing is interpreted in the context of the planned stopping rule. For an upper efficacy boundary, the first look $k$ at which $Z_k\geq b_k$ is the first efficacy crossing. If a crossing occurs, the design typically stops for efficacy; statistics from later looks are not observed as part of that trial’s planned conduct. A sequence of hypothetical values beyond a crossing may be useful for teaching, but it is not a sequence of further trial decisions under that stopping rule.

Worked Example: Crossing at the Second Look

In another hypothetical trial with the same fractions and cutoffs, the first statistic is $Z_1=4.90$, and the second is $Z_2=2.61$. What is the decision?

At the first look, $4.90<5.112$, so the upper efficacy boundary is not crossed. At the second look, $2.61\geq2.556$, so the boundary is crossed. The first efficacy crossing is at the 60% information look, and the trial stops for efficacy under this rule. No final-look statistic is needed to make that decision.

The result is specific to the direction encoded by the upper boundary. A negative statistic of large magnitude would not cross this one-sided upper boundary; it would be relevant to a lower boundary only if the design specified one.

Worked Example: A Statistic Below Every Cutoff

A third hypothetical trial produces $Z_1=5.00$, $Z_2=2.40$, and $Z_3=1.90$. Check all three comparisons using the same design.

At look 1, $5.00<5.112$, so there is no crossing. At look 2, $2.40<2.556$, so there is no crossing. At the final look, $1.90<1.980$, so the final efficacy boundary is also not crossed. Under this efficacy rule, these results do not meet the stopping threshold at any planned analysis.

A failure to cross this efficacy boundary does not establish that $H_0$ is true. It means only that the observed path did not meet the specified criterion for declaring efficacy. Other prespecified rules, such as a futility rule, would be separate decision criteria.

Diagnostics, Comparisons, and Limitations

Before interpreting a crossing, check that the planned information fractions are increasing and that each observed statistic is paired with the correct look. Confirm whether the stated boundary is one-sided or two-sided, whether the statistic has the intended sign, and whether the critical values are based on the design’s actual calibration. Rounding can matter when an observed statistic lies extremely close to a cutoff; use unrounded values for the decision when available, and state the comparison clearly.

The O’Brien–Fleming shape is especially demanding early and comparatively permissive near the end. In contrast, Pocock’s boundary uses a flatter critical-value shape, with more similar thresholds across looks. That distinction is useful for understanding the different timing of the stopping opportunities, but neither shape removes the need to calibrate the boundary for the planned sequence. A single-look test at each interim analysis is not an equivalent substitute: it ignores the repeated opportunities to cross and generally does not control the intended overall type I error rate.

The calculation also depends on the information fractions being meaningful for the analysis. If the assumed information model is poor, the statistic is not adequately approximated by the canonical model, or the schedule or analysis changes after interim results are seen, the supplied cutoffs may no longer have their intended operating characteristics. An O’Brien–Fleming boundary gives a prespecified efficacy decision rule; it does not itself guarantee that the study has adequate power, that an effect is clinically important, or that the model assumptions are satisfied.

Common pitfall: Comparing every observed statistic with the final cutoff, or declaring a crossing because an interim statistic is “almost” at its cutoff, uses the wrong decision rule. Compare $Z_k$ with $b_k=c/\sqrt{t_k}$ at the same look, using the specified inequality $Z_k\geq b_k$. A non-crossing is not a crossing, even when the values are close.

Confirm the design.
Write down the planned information fractions, test direction, sidedness, and calibrated constant or tabulated cutoffs.

Calculate or verify each cutoff.
For the upper O’Brien–Fleming boundary, use $b_k=c/\sqrt{t_k}$ and retain sufficient precision.

Compare at the matching look.
Check whether the observed $Z_k$ is at least $b_k$. Identify the first crossing, if any.

Report the decision in context.
State the look and information fraction, the statistic and cutoff, and whether the efficacy criterion was met.

Check Your Understanding

Use the illustrative cutoffs $5.112$, $2.556$, and $1.980$ at information fractions $0.15$, $0.60$, and $1.00$, respectively.

At the second look, does $Z_2=2.56$ cross the boundary?
A first-look statistic is $Z_1=5.20$. Has the upper efficacy boundary been crossed?
At the final look, $Z_3=1.97$. What is the efficacy decision at that look?
Why should a statistic at the 60% look not be compared with the final-look cutoff?

Solutions. (1) Yes: $2.56\geq2.556$, so the second-look boundary is crossed. (2) Yes: $5.20\geq5.112$, so the first-look boundary is crossed and the efficacy stopping rule is met. (3) No: $1.97<1.980$, so the final boundary is not crossed. (4) Each cutoff is determined by its own information fraction; using the final cutoff at an interim look substitutes a different rule and can give the wrong decision.