Efficacy Futility Rules In Dose Finding

Why Efficacy Futility Must Consider the Remaining Doses

In dose finding, weak efficacy at the dose currently being studied does not by itself show that the trial has no useful path forward. A higher dose may produce clinically meaningful benefit, while a lower dose may offer a better benefit-risk balance. Conversely, an apparent efficacy opportunity at a dose may be irrelevant if its toxicity is unacceptable. An efficacy-futility rule therefore asks a joint question: given the evidence so far, is there a sufficiently plausible remaining dose with meaningful efficacy and acceptable toxicity?

The rule developed here is Bayesian. It uses the posterior distribution from a joint dose-response analysis, which represents uncertainty about efficacy and toxicity probabilities at each dose after observing the trial data. The earlier material on posterior probabilities of acceptable doses provides the underlying probability interpretation; here the focus is how to use that kind of probability to stop for lack of an efficacy opportunity.

Definition: Let $D$ denote the trial data available at a decision time, and let $R$ be the set of doses that remain eligible for allocation. At dose $j$, let $q_j$ be the probability of efficacy by the prespecified assessment time and $p_j$ the probability of dose-limiting toxicity in the defined window. Choose a clinically meaningful efficacy threshold $q_E$ and a maximum acceptable toxicity probability $p_T$. Dose $j$ is a posterior candidate when $q_j \geq q_E$ and $p_j \leq p_T$. The posterior futility probability is $$ F_R(D)=P\left(\bigcup_{j\in R}\{q_j\geq q_E,\ p_j\leq p_T\}\mid D\right). $$ A prespecified rule stops for efficacy futility when $F_R(D)<\gamma_F$, where $\gamma_F$ is the futility boundary.

The thresholds have distinct roles. $q_E$ defines what counts as clinically meaningful efficacy, $p_T$ defines acceptable toxicity, and $\gamma_F$ states how much posterior probability of at least one candidate dose is required to continue. They should be chosen before using the rule to make trial decisions. A small $\gamma_F$ requires strong evidence against a remaining opportunity before stopping; a larger value stops more readily.

What the Posterior Event Does—and Does Not—Mean

The event inside $F_R(D)$ concerns the unknown dose-specific probabilities $q_j$ and $p_j$. Thus $F_R(D)$ is a posterior probability that at least one remaining dose meets both criteria, given the observed data and the specified model and prior. It is not the probability that a future patient will respond, nor is it a probability that the efficacy and toxicity thresholds themselves are correct.

The joint criterion matters. If efficacy and toxicity are modeled together, posterior draws should retain their dependence. A draw in which $q_j$ is high and $p_j$ is also high does not count as a candidate. Calculating separate posterior probabilities for efficacy and toxicity and multiplying them is generally invalid: it assumes posterior independence, which need not hold. Even if separate marginal probabilities are available, the joint probability must account for their association.

The rule also concerns the union of candidate events across remaining doses. The probability that dose 2 qualifies plus the probability that dose 3 qualifies is not generally the probability that at least one qualifies; both events may occur together. For two remaining doses, the exact calculation is

F_R(D)=P(A_2\mid D)+P(A_3\mid D)-P(A_2\cap A_3\mid D),

where $A_j=\{q_j\geq q_E,\ p_j\leq p_T\}$. In practice, posterior draws from the joint model make the calculation direct: for each draw, record whether any remaining dose meets both criteria, then average those indicators.

F_R(D)\approx \frac{1}{B}\sum_{b=1}^{B} I\left\{\text{at least one }j\in R\text{ has }q_j^{(b)}\geq q_E \text{ and }p_j^{(b)}\leq p_T\right\},

Here $B$ is the number of posterior draws and $I\{\cdot\}$ equals 1 when its condition is true and 0 otherwise. This calculation automatically accounts for dependence among doses and between efficacy and toxicity, provided the draws come from a model that represents those relationships adequately.

Worked Example: Combining Candidate Doses Without Double-Counting

Suppose doses 2 and 3 remain. Use $q_E=0.25$, $p_T=0.30$, and $\gamma_F=0.20$. For a transparent hypothetical calculation, suppose the posterior distribution has these three possible joint states:

Probability 0.50: dose 2 has $q_2=0.32,\ p_2=0.18$; dose 3 has $q_3=0.35,\ p_3=0.34$.
Probability 0.25: dose 2 has $q_2=0.22,\ p_2=0.15$; dose 3 has $q_3=0.29,\ p_3=0.24$.
Probability 0.25: dose 2 has $q_2=0.20,\ p_2=0.32$; dose 3 has $q_3=0.21,\ p_3=0.20$.

In the first state, dose 2 qualifies because $0.32\geq0.25$ and $0.18\leq0.30$. Dose 3 does not qualify because its toxicity probability, 0.34, exceeds 0.30. In the second state, dose 3 qualifies: $0.29\geq0.25$ and $0.24\leq0.30$. Dose 2 fails the efficacy criterion. In the third state, neither dose qualifies: dose 2 exceeds the toxicity limit but falls below the efficacy threshold, while dose 3 falls below the efficacy threshold.

The event that at least one remaining dose qualifies therefore occurs in the first two states: $$ F_R(D)=0.50+0.25=0.75. $$ Because $0.75$ is not below the futility boundary $0.20$, the rule does not stop for efficacy futility. This conclusion does not establish that a remaining dose will work; it says the posterior probability of at least one dose meeting both specified criteria is not low enough to trigger this stopping rule.

Accounting for Toxicity and Model Uncertainty

Toxicity can make a seemingly promising efficacy pattern futile. If all remaining doses are likely to exceed $p_T$, the joint candidate probability may be small even when one or more doses have high efficacy probabilities. This is different from a safety stopping rule: a safety rule can stop because toxicity is unacceptably high, whereas the efficacy-futility rule stops because no remaining dose is sufficiently likely to satisfy the combined efficacy and toxicity criteria. Both rules may be specified, but they answer different questions.

Model uncertainty is a further consideration. A joint dose-response analysis may depend on a choice of functional form, association structure, or prior specification. If several models $M_1,\ldots,M_K$ are considered plausible, let $P(M_k\mid D)$ be the posterior probability of model $M_k$, and let $F_R(D,M_k)$ be the candidate probability calculated under that model. The model-averaged probability is

F_R(D)=\sum_{k=1}^{K}P(M_k\mid D)F_R(D,M_k).

This is a weighted average, not the probability from whichever model gives the most favorable result. Model weights must be specified and updated according to the prespecified Bayesian analysis. If there is only one model, the expression reduces to the single-model posterior probability.

Worked Example: A Promising Efficacy Signal at an Unsafe Dose

Suppose doses 3 and 4 remain, with $q_E=0.30$, $p_T=0.25$, and $\gamma_F=0.15$. Consider hypothetical posterior states with the following probabilities:

Probability 0.12: dose 3 has $q_3=0.34,\ p_3=0.22$, so it qualifies.
Probability 0.38: dose 3 has $q_3=0.36,\ p_3=0.31$ and dose 4 has $q_4=0.39,\ p_4=0.34$; both fail the toxicity criterion.
Probability 0.30: dose 3 has $q_3=0.27,\ p_3=0.18$ and dose 4 has $q_4=0.29,\ p_4=0.20$; both fail the efficacy criterion.
Probability 0.20: both remaining doses fall below $q_E$.

Only the first state contains a dose meeting both criteria. Therefore $$ F_R(D)=0.12. $$ Since $0.12<0.15$, the rule stops for efficacy futility. The second state illustrates why an efficacy-only rule can mislead: efficacy probabilities are high there, but both doses exceed the toxicity limit. The decision follows the prespecified joint rule, not an informal impression that efficacy looks promising.

Worked Example: Averaging Across Two Joint Models

Suppose two plausible joint dose-response models have posterior weights $P(M_1\mid D)=0.60$ and $P(M_2\mid D)=0.40$. The posterior probability of at least one acceptable efficacy candidate is 0.30 under $M_1$ and 0.05 under $M_2$. Let $\gamma_F=0.22$. The model-averaged probability is $$ F_R(D)=0.60(0.30)+0.40(0.05)=0.18+0.02=0.20. $$ Because $0.20<0.22$, the model-averaged rule stops for efficacy futility. Looking only at $M_1$ would yield $0.30$, which exceeds the boundary and would not stop. That difference shows how conclusions can depend on model choice; averaging incorporates the specified uncertainty rather than treating the most optimistic model as certain.

Assumptions, Timing, and Diagnostics

A posterior futility calculation is meaningful only under a coherent analysis plan. At minimum, specify the efficacy and toxicity endpoints, their assessment windows, $q_E$, $p_T$, the set $R$ of doses still eligible, the joint model and prior, and $\gamma_F$. The endpoint definitions and data used in the model must match the clinical question. For the simple examples above, assume binary efficacy and toxicity outcomes are fully ascertained for each included patient by fixed assessment windows. If outcomes are delayed or incomplete, a decision based on immature outcomes must use an analysis that handles that timing; treating an unobserved outcome as absence of efficacy or toxicity can distort the posterior.

A dose belongs in $R$ only if it can still be considered under the trial’s allocation and safety rules. Do not include a dose already excluded by a separate safety rule merely to inflate the chance that “some dose” qualifies. Likewise, the rule cannot identify a candidate outside the modeled dose range. These are design definitions, not matters to decide after seeing which choice gives a preferred result.

Useful checks include confirming that every posterior draw applies the same endpoint definitions and thresholds, verifying that toxicity is included in the candidate event, and checking that posterior model weights sum to 1. If Monte Carlo draws are used, the estimated probability should be stable enough for a decision near $\gamma_F$; a value barely on one side of the boundary deserves more numerical precision than a value far from it. Also inspect whether the posterior draws cover the dose-response patterns that are clinically plausible. A model that imposes an unjustified efficacy trend or toxicity relationship may produce a precise-looking but misleading futility probability.

Derivation and justification: For each posterior draw, define $Z^{(b)}=1$ if any $j\in R$ satisfies both $q_j^{(b)}\geq q_E$ and $p_j^{(b)}\leq p_T$, and set $Z^{(b)}=0$ otherwise. The posterior probability of the union event is the posterior expectation of its indicator, $F_R(D)=E(Z\mid D)$. Averaging $Z^{(b)}$ over posterior draws estimates that expectation. With model uncertainty, the law of total probability gives $P(\cup_j A_j\mid D)=\sum_k P(\cup_j A_j\mid D,M_k)P(M_k\mid D)$, which yields the model-averaging formula above.

Common Mistakes and Alternatives

Stopping because the current dose has low efficacy. This ignores possible benefit at other remaining doses. The rule evaluates the union over all eligible remaining doses.
Using an efficacy-only candidate event. A dose with $q_j\geq q_E$ does not count if $p_j>p_T$. Apply both criteria to the same posterior draw.
Adding dose-specific probabilities. Candidate events across doses can overlap. Estimate the probability of their union directly rather than summing marginal probabilities.
Interpreting $F_R(D)$ as a future-patient response rate. It is a posterior probability about dose-level parameters and the candidate event, conditional on the data and model.

A simple alternative is a dose-by-dose rule that stops only when every remaining dose separately has a low probability of meeting the efficacy criterion. That rule may be more conservative or less conservative depending on how it is defined, and it can omit toxicity or dependence if implemented using separate summaries. Another alternative is to use a predictive probability that a future cohort will show enough efficacy. That answers a different question: the present rule assesses the posterior plausibility of a qualifying dose, not the probability that a specified future sample will cross a data threshold. Neither alternative is interchangeable without clarifying the decision target.

No single value of $\gamma_F$ is universally appropriate. A rule can stop too readily if the boundary is high, the efficacy threshold is unrealistic, or the model is too restrictive. It can continue unproductively if the boundary is very low or if toxicity is omitted. Posterior futility is also limited by the dose range, endpoint definitions, prior and model assumptions, and the amount of information accumulated. It is a prespecified decision aid, not proof that no treatment effect exists.

Check Your Understanding

Answer each question, then compare with the solutions below.

There are two remaining doses. Their posterior candidate probabilities are 0.40 and 0.35, and the posterior probability that both are candidates is 0.20. What is the probability that at least one is a candidate?
At a remaining dose, the posterior probability that $q_j\geq q_E$ is 0.70 and the posterior probability that $p_j\leq p_T$ is 0.80. Can their product automatically be used as the joint candidate probability? Why?
With $\gamma_F=0.10$, a calculated $F_R(D)=0.08$ meets the stopping rule. State what this result means and one thing it does not mean.
Two models have posterior weights 0.25 and 0.75, with candidate probabilities 0.40 and 0.12. Calculate the model-averaged candidate probability. Would the rule stop if $\gamma_F=0.20$?

Solutions. (1) By inclusion-exclusion, $0.40+0.35-0.20=0.55$. (2) No. Multiplication assumes posterior independence of the efficacy and toxicity events; their dependence must be modeled or otherwise accounted for. (3) Since $0.08<0.10$, the prespecified rule stops for efficacy futility. It does not mean there is an 8% probability that a future patient responds, or that the null hypothesis is true. (4) The average is $0.25(0.40)+0.75(0.12)=0.10+0.09=0.19$. Since $0.19<0.20$, the rule stops for efficacy futility.