Fleming's Two-Stage Design: A Worked Example

For the worked example:

This quantity is important because it directly determines expected sample size.

In general:

Thus, the expected sample size is a function of the true underlying response probability.

Expected Sample Size Is a Function of \(p\)

There is not one universal expected sample size for a Fleming design.

Instead:

changes as the true response rate changes.

This can be especially useful when evaluating the efficiency of a design across a range of plausible response rates.

Operating Characteristics Across Response Rates

A well-designed Phase II trial should not be evaluated only at \(p_0\) and \(p_1\).

Investigators may also examine:

• Probability of declaring efficacy as a function of \(p\)
• Expected sample size as a function of \(p\)
• Probability of early futility stopping
• Probability of early efficacy stopping
• Probability of reaching Stage 2

The resulting curves are often called the operating characteristics of the design.

Probability of Declaring Efficacy

Define:

At the null response rate:

while at the alternative response rate:

The complete function \(R(p)\) provides substantially more information than two isolated values.

Why the Gray Zone Matters

Suppose:

What happens if the true response rate is 20%?

The design does not promise a specific probability of success at this value.

That is because the design was constructed primarily to control type I error at 10% and achieve adequate power at 30%.

Fleming Versus a Single-Stage Design

A single-stage design might enroll all 29 patients before making its final decision.

Fleming's design has the same maximum sample size in this example but can stop after only 10 patients.

The two-stage design can therefore substantially reduce expected enrollment when the treatment is inactive.

Fleming Versus Simon: A More Important Distinction

The difference between Fleming and Simon is more subtle than simply saying that one has two stages and the other has two stages.

Both designs can stop early for futility.

The defining feature of Fleming's design is that the first stage can also stop for efficacy.

This can produce a smaller expected sample size when the treatment is extremely active, because some trials can terminate after Stage 1.

However, early efficacy stopping may conflict with a desire to obtain a more precise estimate of response rate.

Why Early Efficacy Stopping Can Be Controversial

Suppose the first 10 patients produce five responses.

The observed response rate is:

This is highly encouraging.

But a sample of only 10 patients also has substantial statistical uncertainty.

A decision-maker may therefore reasonably ask:

Is it better to stop because the treatment looks highly active, or should additional patients be enrolled to obtain a more stable estimate?

Fleming's design prioritizes the former possibility. Simon-style designs commonly prioritize continued estimation after promising interim results.

When Fleming's Design May Be Particularly Useful

A Fleming-style design can be attractive when:

• The treatment is being evaluated in a single-arm Phase II setting.
• The endpoint is binary.
• Rapid screening is important.
• Early futility stopping is clinically valuable.
• Early efficacy stopping is also considered useful.
• The maximum sample size must be tightly controlled.
• The operating characteristics can be prespecified before enrollment.

When Another Design May Be Preferable

Fleming's design is not automatically optimal for every Phase II study.

Other approaches may be preferable when:

• The primary endpoint is continuous rather than binary.
• Time-to-event information is central.
• There is a randomized control group.
• The treatment effect is better represented by a continuous parameter.
• Additional patients should be enrolled after promising early results.
• Multiple treatment arms are being compared.
• The trial requires more complex adaptive decision rules.

Exact Enumeration of the Design

Because the response counts are discrete, Fleming's design can be evaluated exactly by enumerating every possible first-stage response count.

For each possible value of \(X_1\):

1. Determine whether the trial stops for futility.
2. Determine whether the trial stops for efficacy.
3. If neither occurs, calculate the probability of crossing the final boundary in Stage 2.

This produces the exact operating characteristics without relying on a normal approximation.

R Implementation

The design can be evaluated directly using the binomial distribution in R.

p0 <- 0.10
p1 <- 0.30

n1 <- 10
n  <- 29
n2 <- n - n1

a1 <- 1
r1 <- 5
r  <- 6

Here:

• a1 is the Stage 1 futility boundary.
• r1 is the Stage 1 efficacy boundary.
• r is the final efficacy boundary.

Calculate Stage 1 Probabilities

stage1_futility <- pbinom(
  a1,
  size = n1,
  prob = p0
)

stage1_efficacy <- 1 - pbinom(
  r1 - 1,
  size = n1,
  prob = p0
)

stage1_continue <- 1 -
  stage1_futility -
  stage1_efficacy

stage1_futility
stage1_efficacy
stage1_continue

For \(p_0=0.10\), the probability of continuing is approximately:

stage1_continue
# approximately 0.2623

Calculate the Type I Error

The following code evaluates every possible Stage 1 response count that leads to Stage 2.

alpha <- stage1_efficacy

for (x1 in (a1 + 1):(r1 - 1)) {

  p_x1 <- dbinom(
    x1,
    size = n1,
    prob = p0
  )

  p_stage2 <- pbinom(
    r - x1 - 1,
    size = n2,
    prob = p0,
    lower.tail = FALSE
  )

  alpha <- alpha + p_x1 * p_stage2
}

alpha

The resulting value is approximately:

alpha
# approximately 0.0473

Calculate Power

power <- 1 - pbinom(
  r1 - 1,
  size = n1,
  prob = p1
)

for (x1 in (a1 + 1):(r1 - 1)) {

  p_x1 <- dbinom(
    x1,
    size = n1,
    prob = p1
  )

  p_stage2 <- pbinom(
    r - x1 - 1,
    size = n2,
    prob = p1,
    lower.tail = FALSE
  )

  power <- power + p_x1 * p_stage2
}

power

The resulting power is approximately:

power
# approximately 0.8052

Calculate Probability of Early Termination

Under any response probability \(p\), early termination occurs when the first stage crosses either boundary.

pet <- function(p) {

  futility <- pbinom(
    a1,
    size = n1,
    prob = p
  )

  efficacy <- 1 - pbinom(
    r1 - 1,
    size = n1,
    prob = p
  )

  futility + efficacy
}

pet(p0)
pet(p1)

At the null response rate:

pet(p0)
# approximately 0.7377

Thus approximately 73.8% of trials terminate after Stage 1 under the null.

Calculate Expected Sample Size

expected_n <- function(p) {

  continue <- 1 -
    pbinom(a1, n1, p) -
    (1 - pbinom(r1 - 1, n1, p))

  n1 + continue * n2
}

expected_n(p0)
expected_n(p1)

The expected sample sizes are approximately:

expected_n(p0)
# approximately 14.98

expected_n(p1)
# approximately 23.31

Creating an Operating Characteristic Function

It is often useful to evaluate the probability of declaring efficacy over an entire range of possible response probabilities.

prob_success <- function(p) {

  result <- 1 - pbinom(
    r1 - 1,
    size = n1,
    prob = p
  )

  for (x1 in (a1 + 1):(r1 - 1)) {

    p_x1 <- dbinom(
      x1,
      size = n1,
      prob = p
    )

    p_stage2 <- pbinom(
      r - x1 - 1,
      size = n2,
      prob = p,
      lower.tail = FALSE
    )

    result <- result +
      p_x1 * p_stage2
  }

  result
}

prob_success(0.10)
prob_success(0.20)
prob_success(0.30)
prob_success(0.40)
prob_success(0.50)

This function gives the probability that the trial will declare efficacy for any specified true response probability.

Interpreting the Operating Characteristic Curve

Suppose the true response probability is substantially below 10%. The probability of declaring efficacy should be very small.

As the true response probability increases, the probability of declaring efficacy increases.

At the two design points:

and:

The function between these points describes the behavior of the design in the gray zone.

Searching for a Fleming Design

In practice, investigators generally do not choose \(n_1\), \(n\), \(a_1\), \(r_1\), and \(r\) by hand.

Instead, candidate integer designs are enumerated and evaluated against the required operating characteristics.

A generic search considers:

• Candidate Stage 1 sample sizes
• Candidate maximum sample sizes
• Candidate Stage 1 futility boundaries
• Candidate Stage 1 efficacy boundaries
• Candidate final efficacy boundaries

For each candidate design, calculate:

• Type I error
• Power
• Probability of early termination
• Expected sample size

Designs that fail the type I error or power constraints are eliminated.

Optimal vs. Minimax Thinking

Two common ways of evaluating two-stage designs are:

Minimax Design

A minimax design attempts to minimize the maximum possible sample size:

This is attractive when controlling the maximum number of exposed patients or the maximum trial duration is especially important.

Optimal Design

An optimal design may instead minimize the expected sample size under a specified response probability, often under the null:

This is attractive when the investigator wants to minimize the average number of patients enrolled when the treatment is ineffective.

These Objectives Can Conflict

A design with the smallest possible maximum sample size does not necessarily have the smallest expected sample size.

Likewise, a design that is highly efficient under the null may have a larger maximum sample size.

Why Integer Constraints Matter

The response count is discrete.

You cannot require:

The boundary must be an integer.

Similarly, the sample size must be an integer.

This discreteness means that the exact operating characteristics can differ from the nominal values.

For example, the target might be:

but the selected integer design might produce:

which is acceptable because it does not exceed the prespecified type I error constraint.

Protocol Considerations

The protocol should clearly state the design before enrollment begins.

At minimum, specify:

• Primary binary endpoint
• Definition of response
• Null response rate \(p_0\)
• Alternative response rate \(p_1\)
• Type I error
• Power
• Stage 1 sample size
• Maximum sample size
• Stage 1 futility boundary
• Stage 1 efficacy boundary
• Final efficacy boundary
• Rules for handling unevaluable patients
• Analysis population
• Timing of response assessment

Unevaluable Patients Require Special Attention

The simple Fleming calculations assume that every enrolled patient contributes an evaluable binary response.

Real trials may contain:

• Patients who withdraw before assessment
• Patients who violate eligibility criteria
• Patients who receive inadequate treatment
• Patients with missing response assessments
• Patients who are lost to follow-up

The protocol should prespecify how these patients affect the primary endpoint and whether replacement is allowed.

Timing of Response Assessment

A two-stage design also depends on when the endpoint is observed.

If response requires several months of follow-up, the nominal Stage 1 enrollment may finish long before the Stage 1 response data are mature.

Therefore, investigators should distinguish between:

• Enrollment time
• Response assessment time
• Database lock
• Stage transition

A design that looks efficient mathematically may not produce a rapid go/no-go decision operationally if response assessment is slow.

Safety Monitoring Is Separate

The Fleming efficacy boundaries should not be interpreted as a complete safety monitoring strategy.

A Phase II protocol may require separate rules for:

• Serious adverse events
• Dose-limiting toxicity
• Excessive treatment-related mortality
• Laboratory abnormalities
• Other predefined safety signals

Those safety rules should be coordinated with the efficacy design but should not be confused with the Fleming response boundaries.

Common Mistakes

1. Confusing Fleming with Simon. Fleming permits early efficacy stopping; Simon's commonly used two-stage framework generally does not.
2. Using ordinary repeated hypothesis tests. The interim and final boundaries must be incorporated into the design.
3. Ignoring the actual type I error. Because the binomial endpoint is discrete, the achieved type I error may differ from the nominal target.
4. Reporting only the maximum sample size. Expected sample size and probability of early termination are important operating characteristics.
5. Changing the boundaries after Stage 1. Doing so invalidates the original operating characteristics unless the adaptation was incorporated into the design.
6. Assuming 80% power applies to every response rate. Power is a function of the true response probability.
7. Ignoring the clinical meaning of \(p_0\) and \(p_1\). The statistical design cannot compensate for poorly chosen clinical thresholds.
8. Failing to specify how unevaluable patients are handled. Missing response assessments can materially affect the effective operating characteristics.
9. Using a Fleming design automatically. The design should be selected based on the scientific and operational goals of the study.

A Practical Fleming Design Workflow

What Should Be Reported in a Statistical Analysis Plan?

For a Fleming two-stage study, the statistical documentation should make the decision rules completely reproducible.

A useful statistical section should state:

• Primary endpoint definition
• Null response probability
• Alternative response probability
• Type I error
• Target power
• Stage 1 sample size
• Stage 2 sample size
• Maximum total sample size
• Stage 1 futility boundary
• Stage 1 efficacy boundary
• Final efficacy boundary
• Expected sample size under relevant response probabilities
• Probability of early termination
• Method used to derive the operating characteristics
• Handling of missing or unevaluable responses

Worked Example Summary

The complete design can be summarized in one table.

The Most Important Concept

The most important conceptual point is that a Fleming design is not simply a smaller clinical trial.

It is a carefully constructed sequential decision procedure in which the probabilities of all possible paths through the trial are controlled.

At Stage 1, the trial can:

• Stop because there is too little activity.
• Stop because there is exceptionally strong activity.
• Continue because the evidence is intermediate.

If it continues, the accumulated response count is evaluated against the final boundary.

The resulting design balances:

• Type I error
• Power
• Maximum sample size
• Expected sample size
• Early futility stopping
• Early efficacy stopping

References

Fleming, T.R. (1982). One-sample multiple testing procedure for phase II clinical trials. Biometrics, 38, 143–151.
Simon, R. (1989). Optimal two-stage designs for phase II clinical trials. Controlled Clinical Trials, 10(1), 1–10.
Fleming, T.R. & Harrington, D.P. (1991). Counting Processes and Survival Analysis. Wiley.
Chen, T.T. (2013). Optimal, minimax, and admissible two-stage designs for phase II clinical trials. Contemporary Clinical Trials.
Rubinstein, L.V., Korn, E.L., Freidlin, B., Hunsberger, S., Ivy, S.P. & Smith, M.A. (2005). Design issues of randomized phase II trials and a proposal for phase II screening trials. Journal of Clinical Oncology.
Zhou, Y., et al. (2020). Optimal, minimax and admissible two-stage design for phase II oncology clinical trials. BMC Medical Research Methodology.