Alpha-Spending Functions: O'Brien-Fleming vs. Pocock

Introduction

A conventional clinical trial typically specifies a single final analysis. The investigators choose a type I error rate, calculate a sample size, enroll the required patients, and then perform the primary hypothesis test once.

A group sequential design allows the accumulating data to be examined at one or more prespecified interim analyses. The trial may then stop early for overwhelming efficacy, futility, safety, or another prespecified reason.

The statistical challenge is immediate: if the investigators repeatedly test the treatment effect at multiple points in time using the ordinary significance threshold, the probability of a false positive becomes larger than the nominal type I error.

Why Repeated Testing Creates a Problem

Suppose a trial has a two-sided type I error of 5% and the investigators look at the data after 50%, 75%, and 100% of the planned information has accumulated.

If each analysis independently uses a two-sided significance level of 0.05, the overall probability of incorrectly rejecting the null hypothesis is greater than 5%.

The problem can be expressed conceptually as:

The analyses are not independent, so simply multiplying or adding the individual error probabilities is not correct. Instead, the joint distribution of the sequential test statistics must be taken into account.

Group sequential methods solve this problem by selecting boundaries that control the probability of crossing an efficacy boundary under the null hypothesis.

What Is Alpha Spending?

An alpha-spending function specifies how much of the total type I error probability has been made available by a particular information fraction.

Let:

• \(\alpha\) = the overall type I error rate.
• \(t\) = the information fraction at the current analysis.
• \(\alpha(t)\) = cumulative alpha spent by information fraction \(t\).

The spending function must satisfy:

Thus, at the beginning of the study no type I error has been spent, while by the final analysis the entire prespecified type I error budget has been allocated.

Information Fraction

The horizontal axis of an alpha-spending function is usually the information fraction, not simply calendar time.

For a simple fixed-sample trial, the information fraction can often be approximated by the fraction of the planned sample size that has been observed:

For time-to-event endpoints, however, information is usually more closely related to the number of observed events than to the number of enrolled patients.

For example, if the final analysis is planned after 400 events, then an interim analysis after 200 events corresponds approximately to:

From Alpha Spending to a Statistical Boundary

Suppose the trial is testing a two-sided null hypothesis and the cumulative alpha spent by an interim analysis is \(\alpha(t_i)\).

The corresponding nominal Z-boundary can be obtained from the standard normal distribution.

For a two-sided analysis:

where \(\alpha_i\) is the relevant error probability associated with the boundary.

In practice, the exact boundary is determined jointly with the correlation structure of the sequential test statistics. The spending function determines the cumulative error budget, while the sequential boundary construction translates that budget into critical Z-values.

The O'Brien-Fleming Spending Function

The O'Brien-Fleming approach is characterized by a very conservative interim boundary and a boundary that becomes substantially less stringent as the trial approaches its final analysis.

A commonly used O'Brien-Fleming-type spending function for a two-sided test is:

where \(\Phi\) is the standard normal cumulative distribution function.

At \(t=1\):

As \(t\) approaches zero, the amount of alpha spent becomes extremely small. Consequently, an O'Brien-Fleming-type design requires very strong evidence to stop early.

The Pocock Spending Function

A commonly used Pocock-type alpha-spending function is:

This spending function allocates substantially more alpha early in the trial than the O'Brien-Fleming approach.

At the final analysis:

Thus, the entire type I error budget has been spent by the final analysis.

The practical consequence is that the Pocock approach tends to produce interim boundaries that are much closer to the final boundary.

O'Brien-Fleming vs. Pocock: The Core Difference

A Two-Look Worked Example

Consider a randomized clinical trial with a two-sided type I error rate of 5%. The investigators plan one interim analysis at 50% information and a final analysis at 100% information.

We will calculate the cumulative alpha spent at the interim analysis using both spending functions.

Step 1: O'Brien-Fleming Alpha Spending

For a two-sided \(\alpha=0.05\) design:

At \(t=0.50\):

Because:

the cumulative alpha spent is approximately:

or approximately 0.56%.

That means the O'Brien-Fleming approach has used only a small fraction of the 5% type I error budget by the halfway point.

Step 2: Pocock Alpha Spending

Now use the Pocock-type spending function:

At \(t=0.50\):

which gives approximately:

or approximately 3.10%.

Thus, by the halfway point, the Pocock-type spending function has made much more of the type I error budget available than O'Brien-Fleming.

Comparing the Spending Directly

What This Means for Interim Boundaries

Suppose the interim analysis occurs at 50% information.

The O'Brien-Fleming approach permits only about 0.56% cumulative two-sided alpha by this point, whereas the Pocock-type spending function permits about 3.10%.

The corresponding two-sided nominal Z thresholds associated with these cumulative alpha values are approximately:

The important point is not simply that the numerical Z-values differ. The difference represents a fundamentally different philosophy about early evidence.

Why O'Brien-Fleming Is Conservative Early

At an early interim analysis, relatively little information has accumulated. An apparently large treatment effect can therefore be driven substantially by random variation.

O'Brien-Fleming addresses this by demanding very strong evidence before allowing an early rejection of the null hypothesis.

The philosophy is:

As information accumulates, the required evidence decreases.

Near the final analysis:

This makes O'Brien-Fleming attractive when early stopping is desirable only in the presence of overwhelming evidence.

Why Pocock Allows More Early Spending

Pocock-type spending takes a different approach.

Instead of reserving almost all of the alpha budget for the final analysis, it makes substantially more alpha available at interim looks.

The philosophy is:

This can be useful when an early demonstration of efficacy would have major scientific, ethical, or operational value.

Incremental Alpha Spending

Cumulative alpha should not be confused with the alpha spent at an individual analysis.

Suppose the cumulative spending at two analyses is:

The additional alpha made available between the first and second analyses is:

Therefore:

So approximately 4.44 percentage points of cumulative alpha are added between the interim and final analyses.

O'Brien-Fleming vs. Pocock: A Practical Comparison

Alpha Spending with More Than Two Analyses

The real advantage of alpha-spending functions becomes apparent when there are multiple interim analyses.

Consider analyses at information fractions:

The spending function can be evaluated at every information fraction.

For each analysis:

1. Calculate cumulative alpha spending.
2. Determine the corresponding sequential boundary.
3. Compare the observed test statistic with that boundary.
4. Stop if the efficacy criterion is crossed.
5. Otherwise continue to the next analysis.

This allows the number and timing of interim analyses to be specified in advance while maintaining control of the overall type I error.

Unplanned Timing of Interim Analyses

One of the major attractions of alpha-spending methods is that the interim analyses can be associated with information fractions rather than requiring perfectly fixed calendar dates.

For example, suppose an interim analysis was originally anticipated at approximately 50% information, but the actual analysis occurs at 47%.

The spending function can be evaluated at:

rather than automatically using the boundary calculated for exactly 50% information.

One-Sided Alpha Spending

Many confirmatory clinical trials use a one-sided type I error rate, particularly when the scientific question concerns whether a treatment improves an outcome in a prespecified direction.

For a one-sided O'Brien-Fleming-type spending function, a commonly used form is:

The corresponding Pocock-type spending function can be written as:

The same conceptual distinction remains: O'Brien-Fleming spends very little alpha early, whereas Pocock-type spending makes considerably more alpha available at early analyses.

A Complete Two-Look Design Summary

R Implementation

The spending functions can be evaluated directly in R.

alpha <- 0.05

# Information fractions
t <- seq(0.05, 1, by = 0.01)

# O'Brien-Fleming-type spending
alpha_obf <- 2 * (
  1 - pnorm(
    qnorm(1 - alpha / 2) / sqrt(t)
  )
)

# Pocock-type spending
alpha_pocock <- alpha *
  log(1 + (exp(1) - 1) * t)

data.frame(
  information_fraction = t,
  alpha_obf = alpha_obf,
  alpha_pocock = alpha_pocock
)

Calculate the Two-Look Example

alpha <- 0.05
t_interim <- 0.50

# O'Brien-Fleming
alpha_obf_interim <- 2 * (
  1 - pnorm(
    qnorm(1 - alpha / 2) /
      sqrt(t_interim)
  )
)

# Pocock
alpha_pocock_interim <- alpha *
  log(
    1 +
    (exp(1) - 1) *
    t_interim
  )

alpha_obf_interim
alpha_pocock_interim

The approximate results are:

alpha_obf_interim
# approximately 0.0056

alpha_pocock_interim
# approximately 0.0310

Calculate Approximate Nominal Z Values

z_obf <- qnorm(
  1 - alpha_obf_interim / 2
)

z_pocock <- qnorm(
  1 - alpha_pocock_interim / 2
)

z_obf
z_pocock

This gives approximately:

z_obf
# approximately 2.77

z_pocock
# approximately 2.16

Visualizing the Spending Functions in R

plot(
  t,
  alpha_obf,
  type = "l",
  lwd = 2,
  xlab = "Information Fraction",
  ylab = "Cumulative Alpha",
  ylim = c(0, alpha)
)

lines(
  t,
  alpha_pocock,
  lwd = 2,
  lty = 2
)

abline(
  h = alpha,
  lty = 3
)

legend(
  "topleft",
  legend = c(
    "O'Brien-Fleming",
    "Pocock-type"
  ),
  lty = c(1, 2),
  lwd = 2,
  bty = "n"
)

The resulting plot illustrates the fundamental difference between the two approaches.

The O'Brien-Fleming curve remains close to zero during the early portion of the trial and accelerates toward the final 5% level near \(t=1\). The Pocock-type curve spends alpha more steadily throughout the study.

Why Alpha Spending Is Not the Same as Simply Dividing Alpha

A common first attempt at handling interim analyses is to divide the total alpha equally across the number of looks.

For example, with four analyses and an overall alpha of 0.05, someone might suggest using:

at each analysis.

That is generally not how a group sequential design is constructed.

The interim test statistics are correlated because they are based on overlapping accumulating data. The appropriate boundaries therefore depend on the joint sequential distribution.

O'Brien-Fleming and Pocock as Design Philosophies

The two approaches can be understood as different answers to one practical question:

O'Brien-Fleming answers:

Pocock answers:

Neither philosophy is universally superior. The appropriate choice depends on the clinical, ethical, operational, and regulatory objectives of the trial.

When O'Brien-Fleming Is Attractive

• The primary goal is to preserve a strong final analysis.
• Early efficacy stopping is possible but should require overwhelming evidence.
• The trial has a relatively small number of interim analyses.
• Investigators want the final significance threshold to remain close to the conventional fixed-sample threshold.
• There is little desire to sacrifice final-analysis efficiency for early stopping.

When Pocock-Type Spending Is Attractive

• Early stopping for efficacy is particularly valuable.
• There are several planned interim analyses.
• The investigators want relatively similar stopping thresholds across looks.
• There is substantial ethical or operational value in stopping a successful trial earlier.
• The trial is designed around frequent monitoring of accumulating information.

Alpha Spending and Futility

Alpha spending primarily addresses control of type I error for efficacy testing. Futility boundaries are a separate design component.

A group sequential trial may therefore have both:

• an efficacy boundary controlling false-positive error; and
• a futility boundary controlling when the trial may stop because success is unlikely or the treatment effect is insufficient.

Futility may be specified as either binding or nonbinding, depending on the design and analysis framework.

A Practical Alpha-Spending Workflow

What Should Be Reported in a Statistical Analysis Plan?

For a group sequential design, the statistical documentation should make the interim monitoring procedure reproducible.

At minimum, the statistical section should describe:

• Primary endpoint
• Null and alternative hypotheses
• Overall type I error
• One-sided or two-sided testing
• Planned interim analyses
• Information fraction for each analysis
• Alpha-spending function
• Sequential efficacy boundaries
• Futility boundaries, if applicable
• Final analysis rule
• Power calculation
• Maximum sample size or maximum number of events
• Expected sample size
• Probability of early stopping
• Method used to calculate the sequential boundaries
• Rules for deviations from the planned information times

The Most Important Concept

The most important conceptual point is that alpha spending does not change the overall type I error target.

Instead, it changes how the error budget is distributed over the course of the trial.

For a two-sided 5% design:

regardless of whether an O'Brien-Fleming-type or Pocock-type spending function is used.

What changes is the path by which the design reaches that final 5%.

References

O'Brien, P.C. & Fleming, T.R. (1979). A multiple testing procedure for clinical trials. Biometrics, 35, 549–556.
Pocock, S.J. (1977). Group sequential methods in the design and analysis of clinical trials. Biometrika, 64(2), 191–199.
Lan, K.K.K. & DeMets, D.L. (1983). Discrete sequential boundaries for clinical trials. Biometrika, 70(3), 659–663.
Jennison, C. & Turnbull, B.W. (2000). Group Sequential Methods with Applications to Clinical Trials. Chapman & Hall/CRC.
Proschan, M.A., Lan, K.K.K. & Wittes, J.T. (2006). Statistical Monitoring of Clinical Trials: A Unified Approach. Springer.