Combining Efficacy and Futility Boundaries

Why Combine Two Predictive Boundaries?

At an interim look in a single-arm trial, the observed responses are only part of the data that will determine the trial's final conclusion. A predictive probability summarizes the chance that the eventual data will meet the prespecified terminal-success rule, conditional on what has been observed so far and the model for the remaining patients. Two thresholds can use that same probability for different purposes: a sufficiently high value can trigger early efficacy, while a sufficiently low value can trigger early futility.

The space between the thresholds is important. It defines the continue region: the current data do not make terminal success sufficiently likely to stop early for efficacy, but they also do not make it sufficiently unlikely to stop for futility. Continuing is a decision to collect more data, not a declaration that the treatment works or cannot work.

Definition: Let \(q_n\) be the predictive probability, at an interim analysis after \(n\) participants have evaluable outcomes, that the trial will satisfy its prespecified terminal-success rule. Choose a lower futility threshold \(L\) and an upper efficacy threshold \(U\), with \(0\leq L<U\leq 1\). The decision rule is:

Early efficacy: stop for efficacy if \(q_n\geq U\).
Early futility: stop for futility if \(q_n\leq L\).
Continue: continue enrollment or follow-up if \(L<q_n<U\).

The rule assumes a specified terminal analysis, a defined terminal-success event, a predictive model for unobserved outcomes, and an interim look at which the information needed for that model is available. In the binary-response setting, a common model assumes a constant response probability \(p\) across participants, conditionally independent Bernoulli outcomes given \(p\), and a Beta prior \(p\sim\mathrm{Beta}(a,b)\). The usual beta-binomial calculation also assumes that the number of remaining participants and the trial's terminal rule are fixed as specified. If outcomes are pending, participants differ in response probability, or the planned sample size changes, this simple calculation may not describe the actual decision problem.

From a Predictive Probability to Three Regions

Suppose \(N\) participants are planned in total and \(n\) have evaluable outcomes at the interim look. Write \(y\) for the observed number of responses and \(M=N-n\) for the number of remaining participants. If terminal success means at least \(r\) responses among all \(N\) participants, let \(K\) be the number of responses among the remaining \(M\). The interim predictive probability is the probability that \(y+K\geq r\), given the observed data and the model.

Under the Beta-Bernoulli assumptions, the posterior distribution after observing \(y\) responses in \(n\) participants is \(p\mid y,n\sim\mathrm{Beta}(a+y,b+n-y)\). Integrating over this posterior gives the beta-binomial predictive distribution for \(K\):

P(K=k\mid y,n) = \binom{M}{k} \frac{\mathrm{B}(a+y+k,\ b+n-y+M-k)} {\mathrm{B}(a+y,\ b+n-y)}, \qquad k=0,\ldots,M.

Here \(\mathrm{B}(\cdot,\cdot)\) is the beta function. The formula follows by conditioning on \(p\), then averaging the binomial likelihood for \(K=k\) over the posterior density of \(p\):

\begin{aligned} P(K=k\mid y,n) &=\int_0^1 \binom{M}{k}p^k(1-p)^{M-k} \frac{p^{a+y-1}(1-p)^{b+n-y-1}} {\mathrm{B}(a+y,b+n-y)}\,dp\\ &=\binom{M}{k} \frac{\mathrm{B}(a+y+k,b+n-y+M-k)} {\mathrm{B}(a+y,b+n-y)}. \end{aligned}

Consequently, for the terminal count rule,

q_n=P(y+K\geq r\mid y,n) =\sum_{k=\max(0,r-y)}^{M}P(K=k\mid y,n),

where the sum is zero if \(r-y>M\), and the probability is one if \(r-y\leq 0\). More generally, if terminal success is defined by another prespecified rule, \(q_n\) is the predictive probability of that event instead. The combined decision rule then partitions the possible values of \(q_n\), not directly the possible response counts.

Key design condition: The thresholds must be ordered, \(L<U\). If \(L\geq U\), the stopping regions can overlap or leave no coherent continuation region. The protocol should also state what happens at equality; the convention above assigns \(q_n=U\) to early efficacy and \(q_n=L\) to early futility.

Worked Beta-Binomial Decisions

The following examples use the same hypothetical design to show all three regions. Assume \(N=12\), a single interim look after \(n=6\) evaluable participants, a \(\mathrm{Beta}(1,1)\) prior, and terminal success defined as at least \(r=7\) responses. Thus \(M=6\), and the future response count has a beta-binomial distribution. Let \(L=0.10\) and \(U=0.80\). These numbers are illustrative design choices, not universal thresholds.

Worked Example: Early Futility

Suppose there are \(y=2\) responses among the first six participants. The posterior is \(\mathrm{Beta}(3,5)\). Terminal success requires at least five more responses among the six remaining participants, so \(q_n=P(K\geq5\mid y=2,n=6)\).

For this posterior, the beta-binomial probability of six future responses is \(P(K=6)=\frac{(3)(4)(5)(6)(7)(8)}{(8)(9)(10)(11)(12)(13)} =\frac{20160}{1235520}\approx0.01632\). The adjacent probability is obtained from the beta-binomial probability ratio: \(P(K=5)=P(K=6)\frac{30}{8}\approx0.06119\).

Therefore, \(q_n=P(K=5)+P(K=6)\approx0.06119+0.01632=0.07751\). Since \(0.07751\leq L=0.10\), the rule stops early for futility. This means the modeled chance of meeting the terminal response-count rule is at most the prespecified futility threshold; it does not mean that the response probability is zero.

Worked Example: Continue

Now suppose \(y=4\) of the first six participants respond. The posterior is \(\mathrm{Beta}(5,3)\), and at least three additional responses are needed for terminal success. Thus \(q_n=P(K\geq3\mid y=4,n=6)\).

Using the beta-binomial formula, the first three probabilities are \(P(K=0)=\frac{(3)(4)(5)(6)(7)(8)}{(8)(9)(10)(11)(12)(13)}\approx0.01632\), \(P(K=1)=P(K=0)\frac{30}{8}\approx0.06119\), and \(P(K=2)=P(K=1)\frac{15}{7}\approx0.1311\). The remaining probability is the probability of three or more future responses:

q_n=1-\{P(K=0)+P(K=1)+P(K=2)\} =1-(0.01632+0.06119+0.1311) \approx0.7914.

This is above \(L=0.10\), so the futility rule does not stop the trial, but it is below \(U=0.80\), so the efficacy rule does not stop it either. The decision is to continue. The value \(0.7914\) is a predictive probability under the stated model and terminal rule, not a probability that the treatment is effective in some broader clinical sense.

Worked Example: Early Efficacy

Suppose instead that \(y=5\) of the first six participants respond. The posterior is \(\mathrm{Beta}(6,2)\). Two or more responses among the remaining six are needed, so \(q_n=P(K\geq2\mid y=5,n=6)\).

For this posterior, \(P(K=0)=\frac{(2)(3)(4)(5)(6)(7)}{(8)(9)(10)(11)(12)(13)} =\frac{5040}{1235520}\approx0.004079\). The adjacent probability is \(P(K=1)=P(K=0)\frac{36}{7}\approx0.02098\). Hence \(q_n=1-\{P(K=0)+P(K=1)\}\approx1-(0.004079+0.02098)=0.9749\).

Since \(0.9749\geq U=0.80\), the rule stops early for efficacy. Under this design, the observed data and the model imply a high predictive probability of reaching the defined terminal response count. Early efficacy is still a protocol decision based on that predictive criterion; it does not guarantee that every later or alternative analysis would conclude success.

How the Combined Rule Works in a Design

A combined rule should be specified before the interim data are examined. The protocol should make the terminal-success event, prior, interim sample size, thresholds, and equality conventions explicit. Once \(q_n\) is calculated, the decision is mechanical: compare it with both thresholds. The following sequence helps keep the calculation and the action distinct.

Fix the target event.
State exactly what outcome at the planned terminal analysis counts as success, such as at least \(r\) responses among \(N\) participants.

Update with interim data.
Use the observed number of responses and nonresponses to obtain the posterior distribution under the prespecified model.

Predict the terminal event.
Calculate the probability that the future data, combined with the observed data, will satisfy the target event.

Apply both cutoffs.
Compare the predictive probability with \(L\) and \(U\), and stop for futility, stop for efficacy, or continue according to the stated inequalities.

The combination matters because a one-sided rule leaves one type of interim decision unspecified. A futility-only rule can identify a low predictive probability but does not by itself define when to stop early for efficacy. An efficacy-only rule has the corresponding limitation for futility. With both thresholds in place, the continuation region is explicit rather than an informal default. The ordering \(L<U\) ensures the two stopping conditions cannot both hold at the same interim value.

Approach	Interim quantity or rule	What it does not establish by itself
Combined predictive boundaries	Compare the chance of terminal success with both \(L\) and \(U\).	It does not remove dependence on the predictive model or terminal rule.
Futility-only predictive rule	Stop when the predictive probability is sufficiently low.	It does not define an early-efficacy stopping region.
Efficacy-only predictive rule	Stop when the predictive probability is sufficiently high.	It does not define a futility stopping region.
Fixed interim response-count cutoff	Stop according to whether the observed count crosses a specified boundary.	It does not directly express the probability of meeting the terminal rule under a posterior predictive model.
Posterior probability for the current response rate	Evaluates a posterior statement about \(p\), such as whether \(p\) exceeds a reference value.	It is not the predictive probability of satisfying a future terminal-data criterion.

Diagnostics, Pitfalls, and Limitations

Several basic checks can catch errors before a stopping decision is made. Confirm that \(0\leq y\leq n\), that \(M=N-n\) is the number of participants actually remaining under the planned design, and that the terminal event is represented correctly. Check that the posterior parameters are \(a+y\) and \(b+n-y\), and that the predictive probabilities over all \(k=0,\ldots,M\) sum to one. Finally, verify \(0\leq L<U\leq1\) and apply the equality convention consistently. In the response-count example, a higher interim response count at the same look should not be coded as requiring more future responses for the same terminal target.

A common mistake is to interpret \(q_n\geq U\) as meaning “the posterior probability that the treatment works is at least \(U\).” That is not what the rule says. Here \(q_n\) is the predictive probability of meeting a specified future terminal criterion, integrating uncertainty about \(p\) and future outcomes. A posterior probability about \(p\) is a different quantity and can lead to a different decision.

Another pitfall is selecting thresholds after seeing the interim data. That makes the decision rule data-dependent in an unplanned way and can undermine the intended operating characteristics of the design. Thresholds are design parameters: their consequences should be evaluated for the planned model and sample size before the trial uses them. The rule also inherits the assumptions of its predictive model. A misspecified response model, an inappropriate prior, or an outdated terminal sample size can make the calculated \(q_n\) a poor basis for action. A threshold alone cannot repair those problems.

Combined boundaries also do not guarantee that every observed interim state falls neatly into a clinically sensible action. Discreteness can make \(q_n\) jump between attainable values, so some nominal threshold values may be crossed only rarely or not at all for a particular look. Even when the formal regions are well defined, the design should be checked for plausible data states and for the practical consequences of stopping. The calculations above describe a single interim look with complete binary outcomes; other data structures require a predictive model appropriate to those data.

Check Your Understanding

Answer each question using the stated combined-boundary rule. Solutions follow the questions.

If \(L=0.12\), \(U=0.75\), and the interim predictive probability is \(0.58\), what is the decision?
If the predictive probability is \(0.09\) and \(L=0.10\), what is the decision, and what does it mean?
With \(N=10\), \(n=7\), \(y=5\), and terminal success defined as at least six responses, how many additional responses are needed for success, and how many participants remain?
Why is a choice \(L=0.35\), \(U=0.30\) invalid for this three-region rule?
If \(L=0.10\), \(U=0.80\), and \(q_n=0.78\), what happens? What if only \(U\) is changed to \(0.75\)?

Solutions. 1. Since \(0.12<0.58<0.75\), continue. 2. Since \(0.09\leq0.10\), stop for futility; the model assigns at most a 10% predictive probability to meeting the specified terminal-success rule. It does not prove the treatment is ineffective. 3. There are \(M=N-n=10-7=3\) participants remaining, and \(6-y=6-5=1\) additional response is needed. 4. The lower threshold is not below the upper threshold, so the stopping regions can overlap and there is no properly ordered continuation interval. 5. At \(q_n=0.78\), continue because \(0.10<0.78<0.80\). With \(U=0.75\), stop for efficacy because \(0.78\geq0.75\).