Simon’s Two-Stage Design: A Worked Example

$$ H_0:p\le p_0 \qquad\text{vs.}\qquad H_1:p\ge p_1 $$

$$ p_1>p_0 $$

$$ \text{Is there enough activity to justify continuing?} $$

$$ X_1\le r_1 \quad\Rightarrow\quad \text{Stop for futility} $$

$$ X_1>r_1 $$

the study continues to stage 2.

After all evaluable patients have been assessed, let \(X=X_1+X_2\) be the total number of responses. The final success rule is:

Otherwise:

A Complete Worked Example

Consider an early-phase single-arm oncology trial in which the primary endpoint is objective response. Suppose historical data suggest that a response rate of 10% or lower would not justify further development, while a response rate of 30% or higher would be considered sufficiently promising.

Design Parameter	Value
Unacceptable response rate, \(p_0\)	10%
Promising response rate, \(p_1\)	30%
One-sided type I error target	5%
Target power	90%
Primary endpoint	Binary response

The statistical hypotheses are:

For this tutorial, we will use the following illustrative feasible two-stage design:

Parameter	Design
Stage 1 sample size	18
Stage 1 futility boundary	Stop if ≤2 responses
Stage 2 additional sample size	36
Maximum total sample size	54
Final success criterion	>10 responses among 54

This design will be evaluated directly using the binomial distribution. The point is to understand how the operating characteristics are calculated, rather than to treat the example as a universal recommended design for a real clinical protocol.

Step 1: Define the Stage-1 Rule

The first stage enrolls:

The trial stops if two or fewer responses are observed:

Therefore, the trial proceeds to stage 2 only if:

This is the key futility mechanism. The investigators do not need to wait for all 54 patients if the first 18 provide insufficient evidence of activity.

Step 2: Calculate the Probability of Early Stopping

Under a true response rate of 10%, the stage-1 number of responses follows:

The probability of stopping for futility is:

This probability is approximately:

Therefore:

So if the treatment truly has only a 10% response rate, the trial stops after stage 1 approximately 73% of the time.

Step 3: Calculate the Probability of Early Stopping Under the Alternative

Now suppose the true response rate is 30%. Then:

The probability of stopping early remains:

which is approximately:

Therefore:

Thus, when the treatment really has a 30% response rate, approximately 94% of trials continue to stage 2.

This is desirable because an effective treatment should usually have a high probability of passing the first-stage futility screen.

Step 4: Define the Final Decision Rule

The maximum total sample size is:

The final rule is to declare the treatment promising if more than 10 responses are observed:

Equivalently:

Notice that the final decision is evaluated only for trials that actually reach stage 2. The final response threshold is therefore part of a sequential decision rule, not an independent one-sample calculation.

Step 5: Calculate the Type I Error

The type I error is the probability of incorrectly declaring the treatment promising when the true response rate is at or below the null value. For a monotonic binomial test, the largest type I error occurs at:

The probability of final success must account for the fact that the trial can stop after stage 1. Under the design:

For this example:

where:

and:

The resulting type I error is approximately:

Thus the design has a type I error of approximately 1.4%, which is comfortably below the target 5%.

Step 6: Calculate the Power

Power is the probability of declaring the treatment promising when the true response rate is the alternative value:

Using the same sequential probability calculation:

The resulting power is approximately:

Therefore, the design has approximately 91.3% power when the true response rate is 30%.

Operating Characteristic	Approximate Value
Null response rate	10%
Alternative response rate	30%
Type I error	1.36%
Power	91.33%
Stage 1 sample size	18
Stage 2 sample size	36
Maximum sample size	54

Step 7: Expected Sample Size

One of the most useful characteristics of Simon’s design is that the maximum sample size is not necessarily the average number of patients enrolled.

The expected sample size is:

Under the null response rate:

Using:

gives:

Thus, when the treatment is ineffective at the null response rate, the expected sample size is only about:

rather than 54.

Under the promising response rate of 30%:

Therefore:

This is exactly what we want from a futility design:

True Response Rate	Probability of Continuing	Expected Sample Size
10%	26.6%	27.6
30%	94.0%	51.8

Why Expected Sample Size Matters

Maximum sample size determines the largest number of patients that could be required. Expected sample size describes how many patients the design is expected to use under a particular true response probability.

These are different quantities.

Quantity	Meaning
Maximum sample size	Largest possible number enrolled
Expected sample size under \(p_0\)	Average enrollment if treatment is ineffective
Expected sample size under \(p_1\)	Average enrollment if treatment is promising
Early stopping probability	Probability that the trial terminates after stage 1

A good Simon design therefore tries to achieve appropriate type I error and power while keeping expected sample size low under clinically uninteresting response rates.

What Does "Simon Optimal" Mean?

Simon described two commonly discussed optimization criteria: minimax and optimal.

Minimax Design

A minimax design minimizes the maximum total sample size:

subject to the type I error and power constraints.

This approach is attractive when limiting the maximum number of patients is the primary concern.

Optimal Design

The optimal design instead minimizes the expected sample size under the null:

subject to the same statistical constraints.

This is attractive when the primary goal is to minimize the average number of patients exposed to an ineffective treatment.

Searching Across Candidate Designs

The design parameters are discrete. For a given choice of:

• \(n_1\)
• \(n_2\)
• \(r_1\)
• \(r\)

the operating characteristics can be calculated exactly. A design search evaluates many possible combinations and retains those satisfying the type I error and power constraints.

Conceptually, the search looks like:

The remaining feasible designs can then be ranked according to the chosen optimization criterion.

Why the Binomial Distribution Is Central

For a binary response endpoint, the number of responses among \(n\) patients follows a binomial distribution:

The probability of exactly \(x\) responses is:

Simon’s design essentially combines binomial probabilities from the two stages to calculate the probability of each possible trial outcome.

This makes the design exact in the sense that its operating characteristics can be calculated directly from the binomial model rather than relying on a normal approximation.

Understanding the Decision Tree

The worked design can be represented conceptually as:

Stage	Observation	Decision
Stage 1	0–2 responses among 18	Stop for futility
Stage 1	3 or more responses among 18	Continue
Stage 2	Total ≤10 responses	Do not declare promising
Stage 2	Total ≥11 responses	Declare promising

The distinction between the two boundaries is important. The first boundary determines whether the trial continues. The second determines whether the treatment ultimately meets the activity criterion.

What Happens If Exactly 2 Responses Are Observed?

The boundary is prespecified as:

Therefore, exactly two responses means that the study stops. There is no statistical discretion to continue because the observed response count "looks close."

This illustrates an important feature of Simon’s design: the stopping rule should be specified before the study begins.

Interim Analysis Is Not the Same as an Unplanned Look

The first stage is an intentionally designed interim analysis. The type I error and power calculations explicitly incorporate the possibility that the trial will stop at that point.

This is different from repeatedly examining accumulating data and deciding whether to stop based on an informal judgment.

The stopping boundary is part of the statistical design itself.

Why Simon’s Design Is Particularly Useful in Phase II Oncology

Early oncology development frequently involves a single experimental treatment and a binary activity endpoint such as:

• Objective response
• Tumor response
• Hematologic response
• Disease control under a prespecified definition

When the treatment is clearly inactive, continuing to enroll additional patients may provide little scientific value. A futility design allows investigators to reduce this unnecessary exposure.

The design is therefore especially attractive when:

• The treatment is experimental
• The disease population is limited
• Patient exposure should be minimized
• A binary activity endpoint is available
• A single-arm design is scientifically appropriate

Limitations of a Single-Arm Simon Design

Simon’s design does not eliminate the limitations of a single-arm trial.

The null response rate must usually come from historical information. If the historical control rate is poorly estimated or the historical population differs substantially from the current study population, the interpretation can be misleading.

For example, changes in:

• Patient selection
• Supportive care
• Assessment criteria
• Subsequent therapies
• Imaging frequency
• Investigator assessment

can make a historical response rate an imperfect comparator.

The Importance of the Endpoint Definition

The response endpoint must be defined before the design is finalized. For example, the protocol should specify:

• What constitutes a response
• How response is assessed
• When assessments occur
• How confirmation is handled
• How unevaluable patients are classified
• Whether investigator or independent review is primary

Changing the endpoint definition after observing stage-1 results can compromise the validity of the design.

Evaluable Patients vs. Enrolled Patients

Simon’s design is usually formulated in terms of the number of patients with evaluable binary outcomes. In practice, however, investigators may need to enroll additional patients because some patients may not be evaluable for the primary endpoint.

The protocol should therefore define how non-evaluable patients are handled.

A simple enrollment inflation calculation might be:

where \(r\) is the anticipated proportion of non-evaluable participants.

However, this should not be treated as an automatic adjustment. The exact handling of non-evaluable subjects must be consistent with the primary endpoint definition and statistical analysis plan.

Type I Error and Power Are Design Properties

The two most important operating characteristics are:

Type I Error

The probability of declaring the treatment promising when the true response rate is no better than the unacceptable rate:

Power

The probability of declaring the treatment promising when the true response rate is at least the target promising rate:

These quantities are determined jointly by the stage sizes and response boundaries.

Operating Characteristics Across the Entire Response-Rate Range

Although the design is usually constructed around \(p_0\) and \(p_1\), it is useful to examine the probability of declaring the treatment promising across a range of true response rates.

Define the operating characteristic:

For a well-behaved design, this probability should increase as the true response rate increases.

True Response Rate	Interpretation
5%	Very unlikely to declare promising
10%	Null boundary
20%	Intermediate region
30%	Target promising rate
40%	Strongly promising activity

The region between \(p_0\) and \(p_1\) is particularly important. There is no requirement that the design have a particular probability of success at every response rate between these two values.

The Gray Zone Between p₀ and p₁

Suppose:

What if the true response rate is 20%?

That value lies between the null and alternative planning points. The study may have a substantial probability of either declaring the treatment promising or failing to do so.

This is not a flaw. It is an inherent feature of a hypothesis test with finite sample size. The design is constructed to control error probabilities at the specified planning points rather than to guarantee a deterministic result at every possible response rate.

Sample Size Is Not the Only Design Consideration

The statistical design should also consider:

• Patient availability
• Expected accrual rate
• Duration of follow-up
• Response assessment timing
• Potential delayed responses
• Safety monitoring
• Reference historical data
• Clinical relevance of the response threshold

For some diseases, waiting for enough patients to become evaluable for response may take substantially longer than expected. The timing of the interim analysis should therefore be considered operationally, not just mathematically.

Simon’s Design and Safety

A futility design should not be interpreted as a substitute for safety monitoring. The statistical stopping rule addresses the efficacy/activity endpoint.

A trial may still require:

• Safety review
• Adverse-event monitoring
• Dose modification rules
• Independent safety oversight
• Protocol-defined stopping criteria for toxicity

Thus, a Simon two-stage efficacy design can coexist with separate safety monitoring procedures.

Sample Size Planning in R

The operating characteristics of the worked example can be calculated directly using the binomial distribution in R.

p0 <- 0.10
p1 <- 0.30

n1 <- 18
n2 <- 36

r1 <- 2
r  <- 10

The following function calculates the probability of ultimately declaring the treatment promising:

success.prob <- function(p, n1, n2, r1, r) {

  x1 <- r1 + 1

  sum(
    dbinom(x1:n1, size = n1, prob = p) *
      pbinom(r - (x1:n1),
             size = n2,
             prob = p,
             lower.tail = FALSE)
  )
}

The type I error is evaluated at the null response rate:

alpha.actual <- success.prob(
  p = p0,
  n1 = n1,
  n2 = n2,
  r1 = r1,
  r = r
)

alpha.actual

This gives approximately:

# 0.01365

The power is evaluated at the alternative response rate:

power <- success.prob(
  p = p1,
  n1 = n1,
  n2 = n2,
  r1 = r1,
  r = r
)

power

which gives approximately:

# 0.9133

Calculating the Probability of Early Stopping in R

The stage-1 continuation probability is:

continue.prob <- function(p, n1, r1) {

  pbinom(
    r1,
    size = n1,
    prob = p,
    lower.tail = FALSE
  )
}

continue.p0 <- continue.prob(p0, n1, r1)
continue.p1 <- continue.prob(p1, n1, r1)

continue.p0
continue.p1

The approximate results are:

continue.p0
# 0.2662

continue.p1
# 0.9400

Thus:

• Approximately 26.6% of trials continue under \(p=0.10\).
• Approximately 94.0% continue under \(p=0.30\).

Calculating Expected Sample Size in R

expected.N <- function(p, n1, n2, r1) {

  n1 + n2 * continue.prob(
    p = p,
    n1 = n1,
    r1 = r1
  )
}

expected.N(p0, n1, n2, r1)
expected.N(p1, n1, n2, r1)

The results are approximately:

# 27.58
# 51.84

This provides a compact summary of the efficiency gained by early stopping.

Searching for a Simon Design in R

A simple design search can evaluate candidate combinations of stage sizes and response boundaries. For example:

p0 <- 0.10
p1 <- 0.30

alpha.target <- 0.05
power.target <- 0.90

designs <- list()

k <- 1

for (n1 in 5:50) {

  for (n2 in 1:60) {

    for (r1 in 0:(n1 - 1)) {

      for (r in r1:(n1 + n2 - 1)) {

        alpha.actual <- success.prob(
          p0, n1, n2, r1, r
        )

        power.actual <- success.prob(
          p1, n1, n2, r1, r
        )

        if (
          alpha.actual <= alpha.target &&
          power.actual >= power.target
        ) {

          designs[[k]] <- data.frame(
            n1 = n1,
            n2 = n2,
            N = n1 + n2,
            r1 = r1,
            r = r,
            alpha = alpha.actual,
            power = power.actual,
            EN0 = expected.N(
              p0, n1, n2, r1
            )
          )

          k <- k + 1
        }
      }
    }
  }
}

designs <- do.call(rbind, designs)

The resulting candidate designs can then be sorted according to the desired optimization criterion.

For a minimax criterion:

designs.minimax <- designs[
  order(designs$N, designs$EN0),
]

head(designs.minimax)

For an optimal design emphasizing expected sample size under the null:

designs.optimal <- designs[
  order(designs$EN0, designs$N),
]

head(designs.optimal)

Why Design Search Is Necessary

There is generally no simple closed-form equation that directly gives all four design parameters:

Instead, candidate integer designs are evaluated until the type I error and power requirements are satisfied.

This is fundamentally different from a simple parallel-group sample size formula where the sample size may be calculated from a direct algebraic expression.

Simon’s design is therefore best thought of as a discrete optimization problem subject to statistical constraints.

Common Mistakes in Simon Designs

1. Choosing the null response rate arbitrarily. The value of \(p_0\) should have a defensible clinical and historical basis.
2. Choosing the promising response rate too close to the null. A smaller difference between \(p_0\) and \(p_1\) generally requires more information.
3. Ignoring the stage-1 stopping rule when calculating power. The sequential structure must be incorporated into the operating characteristics.
4. Using the final sample size as though all patients are always enrolled. The design may stop early.
5. Confusing optimal and minimax designs. They optimize different criteria.
6. Changing the response boundary after observing stage-1 data. The boundary should be prespecified.
7. Ignoring endpoint evaluability. The statistical design usually assumes an evaluable binary endpoint.
8. Using historical response rates without checking comparability. Historical-control assumptions can be a major source of uncertainty.
9. Treating the interim analysis as a license for informal decision-making. The prespecified rule should determine whether the trial continues.
10. Failing to validate the final design. The operating characteristics should be independently reproduced before the design is finalized.

When Simon’s Two-Stage Design May Not Be Appropriate

Simon’s design is powerful for a particular class of problems, but it is not a universal solution. It may be inappropriate when:

• A randomized concurrent control group is required
• The primary endpoint is time-to-event
• The endpoint is continuous rather than binary
• Response assessment is substantially delayed
• Multiple treatment arms require simultaneous adaptation
• Safety and efficacy decisions are tightly coupled
• The clinical question requires estimation rather than a binary go/no-go decision

Other group-sequential, adaptive, Bayesian, or multi-stage designs may be more appropriate depending on the scientific question.

Simon’s Design vs. a Conventional Fixed-Sample Single-Arm Trial

Feature	Fixed-Sample Design	Simon Two-Stage Design
Stages	One	Two
Early futility stopping	Usually absent	Built into design
Maximum sample size	Fixed	Prespecified
Expected sample size	Usually equal to planned N	Can be substantially lower
Type I error	Based on fixed design	Accounts for sequential rule
Power	Fixed-sample calculation	Accounts for early stopping

The main efficiency advantage comes from avoiding unnecessary second-stage enrollment when early results indicate inadequate activity.

How to Think About the Design Clinically

A useful clinical interpretation is:

The first question is intentionally conservative. The trial should not continue merely because one or two responses occurred.

The second question is more demanding. Even if the treatment passes the first-stage screen, enough total evidence must be accumulated to satisfy the final response criterion.

What the Worked Example Teaches

The example can be summarized in four numbers:

Quantity	Value
Stage 1	18 patients
Stage 1 stopping rule	Stop if ≤2 responses
Maximum sample size	54 patients
Final success rule	≥11 responses

Under the null response rate of 10%, approximately 73% of trials terminate after only 18 patients. The expected sample size is approximately 27.6 patients.

Under the promising response rate of 30%, approximately 94% of trials proceed to stage 2, and the expected sample size is approximately 51.8 patients.

The design therefore concentrates additional enrollment in scenarios where the treatment appears sufficiently active while avoiding much of that enrollment when the treatment is ineffective.

Protocol-Level Sample Size Considerations

A clinical trial protocol using Simon’s two-stage design should clearly specify:

• The primary binary endpoint
• The null response rate \(p_0\)
• The alternative response rate \(p_1\)
• The type I error criterion
• The desired power
• Stage 1 sample size
• Stage 1 futility boundary
• Stage 2 sample size
• Final response threshold
• Maximum total sample size
• Expected sample size under relevant response rates
• Rules for unevaluable patients
• Response assessment methodology
• Timing of the interim analysis
• Safety monitoring procedures

The protocol should also make clear whether the design is intended to minimize maximum sample size or expected sample size under the null.

Validation Before Implementation

A final design should be independently checked before it is incorporated into a protocol or statistical analysis plan. At minimum, verify:

• Stage 1 sample size
• Stage 1 futility boundary
• Stage 2 sample size
• Final response boundary
• Type I error
• Power
• Expected sample size under the null
• Expected sample size under the alternative
• Operating characteristics across intermediate response rates

The exact definitions of the boundaries should also be confirmed because off-by-one errors are surprisingly easy to introduce.

Summary

Simon’s two-stage design is a structured approach for conducting single-arm trials with a binary efficacy or activity endpoint. Instead of enrolling the maximum sample size immediately, the study begins with a smaller first stage.

If the first-stage response count is too low, the trial stops for futility. If sufficient activity is observed, additional patients are enrolled. The final response count is then compared with a prespecified threshold.

The basic hypotheses are:

The design is defined by:

where the first-stage boundary determines whether the trial continues and the final boundary determines whether the treatment is declared promising.

For the illustrative example:

The design has a maximum sample size of 54 patients. It stops after stage 1 when there are two or fewer responses and continues when there are at least three responses. At the end of the study, at least 11 total responses are required to declare the treatment promising.

Under a true response rate of 10%, the probability of early stopping is approximately 73.4%, giving an expected sample size of about 27.6 patients. Under a true response rate of 30%, approximately 94.0% of trials continue to stage 2, with an expected sample size of about 51.8 patients.

The example has an actual type I error of approximately 1.36% and power of approximately 91.3%, satisfying the illustrative targets of no more than 5% type I error and at least 90% power.

References

Simon, R. (1989). Optimal two-stage designs for phase II clinical trials. Controlled Clinical Trials, 10(1), 1–10.
Green, S.J. & Dahlberg, S. (1992). Planned versus attained design in phase II clinical trials. Statistics in Medicine, 11(7), 853–862.
Fleming, T.R. (1982). One-sample multiple testing procedure for phase II clinical trials. Biometrics, 38(1), 143–151.
Korn, E.L., McShane, L.M. & Freidlin, B. (2010). Statistical challenges in the evaluation of treatment effects in randomized clinical trials. Journal of Clinical Oncology, 28(18), 2996–3001.
Chen, T.T. (1997). Optimal three-stage designs for phase II cancer clinical trials. Statistics in Medicine, 16(23), 2701–2711.