Interim Efficacy Stopping in a Randomized Trial

Why Interim Efficacy Stopping Needs a Prespecified Rule

Interim efficacy monitoring lets a trial stop before its planned maximum sample size when accumulated evidence is sufficiently strong. The potential benefit is practical: participants may avoid continuing in a trial whose results already support a favorable conclusion, and resources may be redirected. The decision is consequential, however. It uses data more than once, and an early stop can affect how much information is available to estimate the size of the treatment effect.

A Bayesian rule can express the decision in terms of posterior evidence. At each scheduled look, the trial team updates the model using the data available at that time and asks whether the posterior probability of a prespecified efficacy claim exceeds a threshold. The rule is not simply “stop when the result looks persuasive.” The endpoint, model, analysis population, look schedule, efficacy claim, threshold, and any other stopping conditions should be specified before the interim data are examined.

Definition: An interim efficacy stopping rule is a prespecified procedure that evaluates trial data at one or more planned interim analyses and stops the trial for efficacy when a stated condition is met. A posterior efficacy threshold requires a posterior probability, such as \(P(p_T>p_C\mid\text{data})\), to exceed a chosen value. The trial stops for efficacy at the first scheduled look at which the condition is met.

The rule described here concerns evidence for efficacy. It does not replace safety monitoring, oversight by an independent monitoring committee, or the planned interpretation of the trial’s treatment effect. The target claim and analysis must also be clearly defined: for example, superiority in a binary response probability at a fixed assessment time. The treatment-policy estimand after treatment discontinuation is a separate issue.

A Beta-Binomial Rule for a Binary Endpoint

Suppose participants are randomized to treatment or control and each participant has a binary outcome, such as response or no response, assessed at a prespecified time. At a given interim look, let \(n_T\) and \(n_C\) be the numbers with observed outcomes, and let \(y_T\) and \(y_C\) be the numbers of responses. The model treats outcomes as independent within and between arms, conditional on the arm-specific response probabilities \(p_T\) and \(p_C\). It also assumes that the observed outcomes are sufficiently mature and that the response definition is applied consistently.

For a simple illustration, assign independent uniform priors: \(p_T\sim\text{Beta}(1,1)\) and \(p_C\sim\text{Beta}(1,1)\). With a binomial likelihood in each arm, the posterior distributions at a look are

p_T\mid\text{data}\sim\text{Beta}(1+y_T,\ 1+n_T-y_T),\qquad p_C\mid\text{data}\sim\text{Beta}(1+y_C,\ 1+n_C-y_C).

For this example, define efficacy as \(p_T>p_C\), and stop when \(P(p_T>p_C\mid\text{data})>0.99\). The value 0.99 is an illustrative design choice, not a universal standard. A different trial might define efficacy using a clinically meaningful margin, such as \(p_T-p_C>\delta\), or use a different prior. Such choices change the posterior probability and must be settled in advance.

Key distinction: The posterior probability is conditional on the model, prior, and data available at that look. A threshold crossing is evidence under those assumptions; it is not the probability that the treatment works, a guarantee that the observed effect is clinically important, or automatic control of a frequentist type I error rate.

Worked Examples of Repeated Looks

Worked Example: First Crossing With a Binary Response

Consider a hypothetical randomized trial with the uniform priors and 0.99 threshold just defined. At the first look, 2 of 3 treatment participants and 0 of 3 control participants have responded. Calculate the posterior efficacy probability and determine whether to stop.

The posteriors are \(p_T\mid\text{data}\sim\text{Beta}(3,2)\) and \(p_C\mid\text{data}\sim\text{Beta}(1,4)\). For a \(\text{Beta}(1,4)\) variable, the cumulative distribution function at \(t\) is \(1-(1-t)^4\). Therefore,

P(p_T>p_C\mid\text{data}) =E\{P(p_C<p_T\mid p_T,\text{data})\} =1-E\{(1-p_T)^4\} =1-\frac{(2)(3)(4)(5)}{(5)(6)(7)(8)} =\frac{13}{14} =0.9286.

Because \(0.9286<0.99\), the trial does not stop for efficacy at this look. At the next scheduled look, suppose the cumulative data are 5 of 5 responses in the treatment arm and 0 of 5 in the control arm. These totals are consistent with the first look: the additional two treatment participants responded, while the two additional control participants did not.

The new posteriors are \(\text{Beta}(6,1)\) and \(\text{Beta}(1,6)\). Using the control posterior cumulative distribution function \(1-(1-t)^6\),

P(p_T>p_C\mid\text{data}) =1-E\{(1-p_T)^6\} =1-\frac{1\cdot2\cdot3\cdot4\cdot5\cdot6}{7\cdot8\cdot9\cdot10\cdot11\cdot12} =\frac{923}{924} =0.9989.

This exceeds 0.99, so the first efficacy stopping boundary crossing occurs at the second look. The decision is supported by a posterior probability of 0.9989 that the treatment response probability exceeds the control response probability, under the specified model and prior. It does not, by itself, quantify the probability that the treatment effect exceeds a clinically important margin.

Worked Example: Strong Evidence That Does Not Cross the Boundary

At a hypothetical look, suppose 4 of 5 treatment participants and 1 of 5 control participants have responded. Use the same model and 0.99 threshold. The posteriors are \(p_T\sim\text{Beta}(5,2)\) and \(p_C\sim\text{Beta}(2,5)\). For the control posterior, integration of its density gives \(F_C(t)=15t^2-40t^3+45t^4-24t^5+5t^6\). Thus,

\begin{aligned} P(p_T>p_C\mid\text{data}) &=\int_0^1 30t^4(1-t)F_C(t)\,dt\\ &=30\left[ 15\left(\frac{1}{7}-\frac{1}{8}\right) -40\left(\frac{1}{8}-\frac{1}{9}\right) +45\left(\frac{1}{9}-\frac{1}{10}\right) -24\left(\frac{1}{10}-\frac{1}{11}\right) +5\left(\frac{1}{11}-\frac{1}{12}\right) \right]\\ &=0.9600\ \text{(approximately)}. \end{aligned}

This is substantial posterior evidence of superiority, but it is below 0.99. The prespecified rule therefore says not to stop for efficacy at this look. A decision-maker should not lower the threshold after seeing these data merely because the result seems persuasive. The threshold and its operating characteristics are design choices, not adjustments to make in response to interim results.

Worked Example: A Continuous Outcome at Two Looks

A different hypothetical randomized trial measures a continuous outcome for which higher values are better. Assume independent normally distributed outcomes in the two arms with a known common standard deviation of 10 units. Let \(\theta\) be the treatment-minus-control mean difference, with prior \(\theta\sim N(0,100)\). At each look, the observed difference \(d\) has likelihood \(d\mid\theta\sim N(\theta,v)\), where \(v=10^2(1/n_T+1/n_C)\). Use the same efficacy threshold, \(P(\theta>0\mid\text{data})>0.99\).

At the first look, \(n_T=n_C=20\) and \(d=5\) units. The posterior variance is \(V=(1/100+1/v)^{-1}\), and the posterior mean is \(m=Vd/v\). Substituting \(v=100(1/20+1/20)=10\) gives

V=(0.01+0.10)^{-1}=9.091,\qquad m=9.091(5/10)=4.545,\qquad P(\theta>0\mid\text{data})=\Phi\left(\frac{4.545}{\sqrt{9.091}}\right)=\Phi(1.508)=0.9342.

There is no efficacy stop at the first look. At the second look, suppose \(n_T=n_C=40\) and the cumulative mean difference is 6 units. This can follow the first look, for example, if the mean difference among the newly observed participants is 7 units. Now \(v=100(1/40+1/40)=5\), so

V=(0.01+0.20)^{-1}=4.762,\qquad m=4.762(6/5)=5.714,\qquad P(\theta>0\mid\text{data})=\Phi\left(\frac{5.714}{\sqrt{4.762}}\right)=\Phi(2.619)=0.9956.

The posterior probability exceeds 0.99, so the trial crosses the efficacy boundary at the second look. This calculation relies on the stated known-variance normal model and prior; with an estimated variance or a different outcome model, the posterior calculation would differ. The example illustrates the same first-crossing logic, not a rule that can be transferred to every endpoint without checking its assumptions.

Why the First-Crossing Rule Must Be Calibrated

A threshold specifies what posterior evidence is sufficient at a single look. Repeated looks create a design whose overall behavior depends on more than that threshold: the number and timing of looks, enrollment and outcome accrual, prior distributions, stopping conditions, and the true response rates all matter. It is therefore not correct to assume that a 0.99 posterior threshold automatically provides a particular frequentist false-positive rate.

A useful design evaluation simulates complete trials under prespecified scenarios. Each simulated trial follows the actual randomization, data-availability schedule, model updates, and stopping rule. Under a null or clinically uninteresting scenario, estimate how often the rule stops for efficacy; under plausible beneficial scenarios, estimate the chance and timing of stopping. Examine sensitivity to reasonable prior choices and to departures from the planned data-generating assumptions. The earlier material on calibrating posterior thresholds by simulation and multiplicity in a three-arm Bayesian trial addresses related calibration issues; here, the essential point is that the repeated-look rule must be evaluated as a whole.

The calculation behind a first-crossing decision is straightforward. At look \(k\), calculate the posterior probability \(q_k\) of the prespecified efficacy claim. Stop if \(q_k\) exceeds the boundary; otherwise continue to the next planned look, if one remains. A complete report identifies the look, the data then available, the posterior probability and boundary, and the decision. If the boundary is not crossed by the final look, report that outcome rather than describing a near miss as a stop.

Prespecify the decision.
Define the efficacy claim, model, prior, threshold, planned looks, and any additional stopping requirements before reviewing interim results.

Check the interim data.
Confirm that outcome definitions and follow-up are consistent, and that the data available for the look meet the planned analysis rules.

Update and compare.
Compute the posterior probability of the efficacy claim and compare it with the prespecified boundary.

Apply the first-crossing rule.
Stop for efficacy at the first boundary crossing; otherwise continue according to the protocol and reassess only at planned looks.

Report the evidence and context.
Give the observed data, model-based posterior result, stopping decision, and relevant design calibration and limitations.

Diagnostics, Alternatives, and Limitations

Interim monitoring can fail operationally even when the mathematics is correct. If response assessments are incomplete or delayed, an apparent difference in observed response rates may reflect different follow-up rather than different response probabilities. Check that the number of participants with mature outcomes is reported by arm and that the planned handling of unavailable outcomes is followed. Also check data integrity, endpoint coding, randomization records, and whether any unplanned analysis has influenced the decision.

The posterior result is conditional on the model. For a binary endpoint, the simple beta-binomial model does not account for clustering, covariate adjustment, or borrowing across groups unless those features are explicitly incorporated. The independent priors and uniform prior used in the examples are convenient illustrations, not mandatory choices. Prior sensitivity is particularly important when interim sample sizes are small. Model diagnostics and sensitivity analyses should be planned at a level appropriate to the trial; a posterior probability reported to four significant figures does not eliminate model uncertainty.

A fixed-sample Bayesian analysis waits until the planned final analysis and avoids interim efficacy decisions, but may continue longer than necessary when evidence becomes compelling early. Predictive futility monitoring asks whether the trial is likely to achieve a target at its end and answers a different question from whether current posterior evidence has crossed an efficacy boundary. A frequentist group-sequential design uses prespecified boundaries designed around frequentist error criteria. These approaches are alternatives with different decision targets and calibration requirements; none should be treated as interchangeable solely because each permits early stopping.

Early stopping also limits the amount of information available about the magnitude and durability of benefit. The posterior probability of superiority can be very high while the effect-size estimate remains imprecise, especially after a small number of outcomes. A decision to stop for efficacy therefore supports a conclusion under the stated model and rule; it does not prove causation beyond the randomized comparison, guarantee clinical importance, or settle safety and regulatory questions.

Common mistake: treating each look as a fresh, isolated test. Repeated decisions are parts of one design. Calibrate and report the full stopping procedure, not just the final posterior probability.
Common mistake: changing the boundary after seeing interim results. This undermines the prespecified decision rule. Use the planned boundary, or treat a changed rule as a design deviation requiring transparent assessment.
Common mistake: equating “posterior probability above threshold” with “clinically important effect.” The claim may concern any positive effect. Define a meaningful margin explicitly if that is the intended decision.
Common mistake: reporting only the stopping decision. Give the observed counts or summaries, model and prior, posterior probability, threshold, analysis look, and relevant calibration so readers can interpret the evidence.

Key takeaway: Interim efficacy stopping is a prespecified, repeated decision based on the posterior probability of a defined claim. Stop at the first planned look that crosses the threshold, but interpret the result in light of the model, data maturity, prior sensitivity, and simulated operating characteristics of the complete design.

Check Your Understanding

Answer each question, then compare your reasoning with the brief solutions.

With a 0.99 efficacy threshold, a look produces \(P(p_T>p_C\mid\text{data})=0.985\). What does the prespecified rule say?
Why does a 0.99 posterior threshold not, by itself, guarantee a particular frequentist false-positive rate across repeated looks?
In a binary-outcome trial, name two data or model checks that matter before acting on an interim posterior probability.
What should be reported when a trial stops for efficacy at its second planned look?

Solutions. (1) Do not stop for efficacy at that look because \(0.985<0.99\); continue only as specified by the protocol. (2) The overall behavior depends on the repeated-look schedule, model, prior, accrual, and stopping rule, so the complete design must be evaluated, commonly by simulation under relevant scenarios. (3) Examples include checking that outcomes are mature and comparably assessed by arm, verifying endpoint coding and data integrity, and assessing whether the prespecified model is reasonable. (4) Report the look and data available, posterior probability and threshold, model and prior, the first-crossing decision, and relevant calibration and limitations.