O'Brien-Fleming Thresholds Near The Final Analysis

Why the Final Threshold Looks Familiar

An O’Brien-Fleming design is deliberately conservative at early interim analyses: a result must be unusually far from the null hypothesis to stop the trial for efficacy. Yet its final critical value is often close to the cutoff from a fixed-sample test at the same significance level. The reason is the way the boundary changes with information. Early in the study, the standardized statistic is based on less information and its threshold is raised sharply; at the final analysis, the information fraction is one, so that inflation disappears.

“Close” does not mean “identical.” The boundary is calibrated to control the probability of a false efficacy conclusion across all planned looks. Because an early crossing can occur in addition to a final crossing, the calibrated final threshold is generally a little more stringent than the fixed-sample cutoff. The size of that difference depends on the significance level and the planned information fractions.

Definition: Let \(I_k\) be the information at look \(k\), and let \(I_K\) be the planned information at the final look \(K\). The information fraction is \(t_k=I_k/I_K\), with \(0<t_1<\cdots<t_K=1\). An O’Brien-Fleming efficacy boundary has critical values \(b_k=c/\sqrt{t_k}\), where \(c\) is calibrated for the chosen type I error rate and the planned looks. For a two-sided test, efficacy is declared at look \(k\) if \(|Z_k|\ge b_k\); for a one-sided test in the positive direction, the condition is \(Z_k\ge b_k\).

Information Scaling and the Shape of the Boundary

The standard information-scale model explains the shape. Under the null hypothesis, assume the sequence of standardized test statistics follows the canonical joint normal model: each \(Z_k\) has mean zero and variance one, and for \(i<j\), \(\operatorname{Corr}(Z_i,Z_j)=\sqrt{t_i/t_j}\). This model arises when information accumulates through independent increments and the statistic at each look is standardized by its standard error. It is the basis for the usual classical O’Brien-Fleming boundary.

One way to represent this joint distribution is \(Z(t)=B(t)/\sqrt{t}\), where \(B(t)\) is Brownian motion under the null. In particular, \(B(t)\) has mean zero and variance \(t\), so \(Z(t)\) has mean zero and variance one. The representation also gives the stated correlation: for \(s<t\), \(\operatorname{Cov}(B(s),B(t))=s\), and therefore \(\operatorname{Corr}(Z(s),Z(t))=s/\sqrt{st}=\sqrt{s/t}\).

With the boundary \(c/\sqrt{t}\), crossing at time \(t\) is equivalent to crossing a constant boundary on the unstandardized information scale:

Z(t)\ge \frac{c}{\sqrt{t}} \quad\Longleftrightarrow\quad \frac{B(t)}{\sqrt{t}}\ge \frac{c}{\sqrt{t}} \quad\Longleftrightarrow\quad B(t)\ge c.

For a two-sided test, the corresponding equivalence is \(|B(t)|\ge c\). Thus, the O’Brien-Fleming form sets a constant threshold for the accumulated Brownian-scale process; when that process is expressed as a standardized \(Z\)-statistic, the threshold must be divided by \(\sqrt{t}\). At an early look, \(t\) is small and this multiplier is large. At the final look, \(t_K=1\), so \(b_K=c\).

Calibration: For a two-sided design, choose \(c\) so that under \(H_0\), \(P\bigl(\text{at least one planned look has }|Z_k|\ge c/\sqrt{t_k}\bigr)=\alpha\). For a one-sided design, calibrate the probability of at least one crossing in the specified direction to \(\alpha\). The constant depends on the significance level and the schedule of information fractions; it is not automatically the fixed-sample quantile.

Why the Final Cutoff Is Near, but Not Equal to, the Fixed-Sample Cutoff

For a fixed-sample two-sided test with significance level \(\alpha\), the usual critical value is \(z_{1-\alpha/2}\), the standard normal quantile with upper-tail probability \(\alpha/2\). At the final look, the O’Brien-Fleming boundary is \(c\), because \(t_K=1\). Since the group sequential rejection event includes crossing at the final look, a calibrated value \(c\) cannot generally be as low as the fixed-sample cutoff: the earlier looks add opportunities to cross. In a nondegenerate design with at least one earlier look, \(c\) is generally greater than \(z_{1-\alpha/2}\) for a two-sided test (or \(z_{1-\alpha}\) for a one-sided test).

The increase is usually modest because the early critical values are so high. A statistic must be far into the tail to cross early, so early looks use relatively little of the total type I error probability. Most of the opportunity for rejection remains near the final analysis. Consequently, the final cutoff stays close to the fixed-sample cutoff even though the early cutoffs can be much larger.

It is useful to distinguish the boundary’s shape from its calibration. The inverse-square-root rule specifies how the thresholds change across information fractions. The constant \(c\) sets their overall level to achieve the desired familywise type I error rate. Using the fixed-sample quantile in place of a calibrated \(c\) may illustrate the shape, but it does not, by itself, establish control of the error rate across multiple looks.

Worked Example: Calculating a Four-Look Boundary

Consider a hypothetical two-sided design with \(\alpha=0.05\) and planned information fractions \(0.25,\ 0.50,\ 0.75,\ 1.00\). Suppose numerical calibration under the canonical joint normal model gives \(c=2.025\). The fixed-sample two-sided cutoff is approximately \(z_{0.975}=1.960\). Calculate the group sequential thresholds.

At each look, use \(b_k=2.025/\sqrt{t_k}\). At \(t_1=0.25\), \(\sqrt{t_1}=0.5\), so \(b_1=2.025/0.5=4.050\). At \(t_2=0.50\), \(b_2=2.025/\sqrt{0.50}=2.864\). At \(t_3=0.75\), \(b_3=2.025/\sqrt{0.75}=2.338\). At \(t_4=1.00\), \(b_4=2.025/1=2.025\).

The two-sided efficacy rule is \(|Z_k|\ge b_k\) at look \(k\). The final threshold, 2.025, is only 0.065 above the fixed-sample cutoff of 1.960, while the first-look threshold is 4.050. These values illustrate the contrast: a small increase in the final cutoff accompanies a substantial increase in the evidence required at the earliest look. The stated \(c\) is a calibrated value for this illustrative schedule, not a universal constant for every four-look design.

Reading the Boundary in a Trial

A boundary is a stopping rule, not a sequence of independent tests. At each planned look, compare the statistic with the threshold for that look. If it crosses, stop for efficacy according to the design. If it does not cross, continue to the next planned look when continuation is otherwise permitted. Failing to cross an efficacy boundary is not proof of no effect and does not mean that \(H_0\) has been accepted.

The threshold is on the \(Z\)-statistic scale. In a two-sided design, both the positive and negative boundaries matter: \(|Z_k|\ge b_k\) indicates crossing in either direction, although the scientific interpretation depends on the direction of the estimate. A one-sided design instead has a boundary in its prespecified direction. The direction and sidedness must be specified in advance; switching them after seeing interim results changes the testing procedure.

Worked Example: Applying the Boundary to Interim Results

Suppose the hypothetical design in the previous example observes \(Z\)-statistics \(3.20,\ 2.40,\ 2.10,\) and \(2.00\) at its four planned looks. Apply the boundary in order.

At look 1, \(|3.20|<4.050\), so there is no efficacy crossing. At look 2, \(|2.40|<2.864\), so the design continues. At look 3, \(|2.10|<2.338\), so it continues again. At the final look, \(|2.00|<2.025\), so the group sequential rule does not reject \(H_0\).

For comparison, a fixed-sample test at the final look would compare \(2.00\) with \(1.960\) and reject at the 0.05 two-sided level. That is not the decision rule for this sequential design: the final statistic must be judged against the calibrated sequential cutoff. The fixed-sample comparison helps show why the final O’Brien-Fleming threshold is near the familiar cutoff, but it cannot replace the planned group sequential analysis.

Justifying the Difference From a Fixed-Sample Test

The final-look crossing event is part of the overall crossing event. For a two-sided design, this implies

P_{H_0}\!\left(\max_{1\le k\le K}|Z_k|\ge \frac{c}{\sqrt{t_k}}\right) \ \ge\ P_{H_0}(|Z_K|\ge c) =2\{1-\Phi(c)\},

where \(\Phi\) is the standard normal cumulative distribution function. To make the left-hand probability equal to \(\alpha\), it follows that \(2\{1-\Phi(c)\}\le\alpha\), hence \(c\ge z_{1-\alpha/2}\). With earlier looks that can genuinely cross, the inequality is generally strict: there is positive probability of crossing early without crossing at the final look. The one-sided argument is the same, replacing the two-sided tail probability by \(1-\Phi(c)\).

This argument gives the direction of the difference, not its precise size. To obtain the calibrated value, the joint probability across all looks must be evaluated using the planned information fractions and their correlation structure. That calculation is why the final critical value can be close to, but is not simply set equal to, the fixed-sample cutoff.

Worked Example: How Rare Is an Early Crossing Under the Null?

Use the calibrated first-look threshold \(b_1=4.050\) from the hypothetical two-sided design above. Under \(H_0\), the marginal statistic at that look has a standard normal distribution under the canonical model. The probability of crossing either first-look boundary is therefore

P_{H_0}(|Z_1|\ge4.050) =2\{1-\Phi(4.050)\} \approx 0.000051.

This is about 0.0051% at that particular look. It is not the design’s overall type I error rate: later looks also contribute, and the statistics across looks are correlated. The calculation does show why an O’Brien-Fleming design spends relatively little type I error at a very early look: the required statistic is deep in the normal tail.

Assumptions, Diagnostics, and Alternatives

The usual boundary calculation assumes that the test statistics and information fractions are suitable for the canonical information-scale model, and that the planned looks and analysis directions are followed. The information used to define \(t_k\) must represent the information available at that look; sample size fractions are interchangeable with information fractions only when the design justifies that equivalence. The outcome model, variance estimation, and test-statistic construction must also support the assumed null distribution and dependence across looks.

Before applying the thresholds, check that the actual analysis times correspond to the planned information fractions, that the statistic has the intended null distribution, and that the stated test is one-sided or two-sided as designed. If information differs materially from plan, or if looks are added or omitted, the original calibration may no longer provide the claimed error control. A boundary crossing based on a statistic with a miscalibrated standard error is not repaired merely by comparing it with the right numerical threshold.

The main classical contrast is Pocock’s boundary, which uses a roughly constant critical value across looks. That makes early stopping less demanding than under O’Brien-Fleming, while generally requiring a higher final cutoff relative to a fixed-sample test. O’Brien-Fleming instead concentrates its efficacy opportunity near the end, preserving a final threshold close to the fixed-sample value. Neither shape is universally preferable: the choice depends on how much early stopping is valued and how the design’s operating characteristics are intended to be balanced.

A limitation is that a stringent early threshold can make early efficacy stopping uncommon, even when an interim result appears promising by ordinary fixed-sample standards. Also, a calibrated \(c\) is tied to the specified significance level and information schedule; it should not be transferred to a different schedule without recalibration. The boundary controls false-positive efficacy decisions under its assumptions, but it does not by itself address other design questions such as futility stopping or changes to the planned sample size.

Common Mistakes

Using the final cutoff at every look. The O’Brien-Fleming threshold is \(c/\sqrt{t_k}\), so applying \(c\) at an early look ignores the information fraction and makes the rule too permissive.
Calling the final cutoff identical to the fixed-sample value. It is often close, but calibration across multiple looks generally puts it slightly above the fixed-sample cutoff.
Treating each look as a separate fixed-sample test at level \(\alpha\). The crossing rule is calibrated jointly across looks; separate unadjusted tests do not have that guarantee.
Interpreting no crossing as acceptance of \(H_0\). It means only that the efficacy stopping condition was not met at the observed looks.
Confusing sample size fraction with information fraction. They coincide only under conditions that make information proportional to sample size.

Key takeaway: O’Brien-Fleming thresholds scale as \(1/\sqrt{t_k}\). They are high early because little information has accrued, but at the final information fraction the threshold is simply the calibrated constant \(c\), usually only modestly above the corresponding fixed-sample cutoff.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

If \(t=0.36\) and the calibrated constant is \(c=2.04\), what is the O’Brien-Fleming threshold?
For a two-sided test with \(\alpha=0.05\), why is the calibrated final threshold generally at least \(1.960\)?
At an interim look, a statistic exceeds the fixed-sample cutoff but not the O’Brien-Fleming boundary. What is the efficacy decision under the sequential design?
Why does the O’Brien-Fleming threshold become less stringent as information accumulates?
What must be reconsidered if the actual information fractions do not match those used to calibrate the boundary?

Solutions. (1) \(b=2.04/\sqrt{0.36}=2.04/0.60=3.40\). (2) The overall rejection event includes final-look rejection and also allows earlier crossings, so calibration to an overall rate of 0.05 requires a final two-sided tail probability no greater than 0.05; thus \(c\ge z_{0.975}\approx1.960\), usually with strict inequality when earlier crossing is possible. (3) There is no efficacy crossing at that look, so the design does not stop for efficacy on that evidence; the fixed-sample cutoff is not the sequential rule. (4) As \(t\) increases, \(1/\sqrt{t}\) decreases, and \(b_k=c/\sqrt{t_k}\) moves toward \(c\). (5) Check whether the information-scale model and planned look schedule still apply, and recalibrate the boundary if the changed schedule requires it.