Why Efficacy and Safety Need Distinct Decisions
An experimental arm can show promising response evidence while also showing an unacceptable toxicity signal. It can also look safe but have little evidence that its efficacy meets a clinically meaningful target. A useful arm-level rule must preserve these distinctions: evidence of harm can prompt a safety action, while weak efficacy evidence can prompt futility, and sufficiently favorable evidence on both dimensions can support an efficacy-success decision under the protocol.
These actions are not interchangeable. A safety-triggered suspension is a response to possible harm and may pause enrollment while the signal is reviewed. Efficacy futility is a conclusion about the chance that the arm meets its efficacy target, not a safety finding. An efficacy-success decision is likewise not a declaration that the treatment is safe; a safety condition must be met separately. The rule should define each action before examining interim results.
The endpoints, assessment windows, target population, and thresholds must be clinically specified. For example, efficacy might be tumor response by a fixed assessment time, while toxicity might be a specified grade of treatment-related adverse event by a separate time. A patient must have the relevant outcome assessed before being counted as a non-event for that endpoint. The probabilities describe the arm-level rates under the model; they are not the probability that an individual patient will benefit or experience toxicity.
A protocol may use separate posterior-probability gates or a joint posterior-probability gate. These are different rules. Separate gates assess efficacy and toxicity one at a time; a joint gate assesses the posterior probability that both desirable conditions hold simultaneously. No single numerical cutoff is universal. Each cutoff, its inequality direction, and the action it triggers should be prespecified.
A Simple Arm-Level Model and Its Assumptions
For a transparent calculation, suppose efficacy and toxicity are each binary and are observed for \(n_j\) fully assessed participants in arm \(j\). Let \(x_{E,j}\) and \(x_{T,j}\) be the respective event counts. One simple model treats outcomes within each endpoint as exchangeable Bernoulli observations with probabilities \(p_{E,j}\) and \(p_{T,j}\). For illustration, assign independent beta priors \(p_{E,j}\sim\mathrm{Beta}(a_E,b_E)\) and \(p_{T,j}\sim\mathrm{Beta}(a_T,b_T)\), with positive shape parameters. This is a modeling choice, not a requirement of Bayesian decision rules.
The binomial likelihood for an endpoint with \(x\) events among \(n\) assessed participants is proportional to \(p^x(1-p)^{n-x}\). Multiplying by a \(\mathrm{Beta}(a,b)\) prior gives a posterior beta distribution. If the two endpoint priors and likelihoods factorize, their posterior distributions also factorize:
The product in the second expression is valid only when the efficacy and toxicity parameters are independent under the joint posterior. Efficacy and toxicity observed on the same participants may be associated. If the analysis represents that association, the joint probability must be calculated from the joint posterior, not obtained by multiplying its marginal probabilities. Separate marginal decisions can still be specified, but they do not by themselves give the probability that both conditions hold.
For example, the numerical illustrations below use a uniform \(\mathrm{Beta}(1,1)\) prior for each rate, \(p_E^*=0.50\), and \(p_T^*=0.20\). They use illustrative cutoffs \(\gamma_T=0.95\), \(\gamma_F=0.10\), and \(\gamma_J=0.90\). These values are not recommendations: the acceptable rates and cutoffs should reflect the clinical context, trial purpose, and consequences of incorrect decisions.
Worked Posterior Calculations
Worked Example: Joint Evidence Is Not the Same as Efficacy Evidence
Suppose a hypothetical arm has 8 efficacy events and 1 toxicity event among 10 fully assessed participants. Use independent \(\mathrm{Beta}(1,1)\) priors for the two rates, and the illustrative thresholds above. Calculate the efficacy evidence, safety evidence, and joint favorable probability.
The posterior distributions are \(p_E\mid D\sim\mathrm{Beta}(9,3)\) and \(p_T\mid D\sim\mathrm{Beta}(2,10)\). For integer beta parameters, the beta cumulative distribution at \(c\) equals the probability that a \(\mathrm{Binomial}(a+b-1,c)\) variable is at least \(a\). Thus, for \(p_E\sim\mathrm{Beta}(9,3)\), the probability of being above 0.50 is the probability that a \(\mathrm{Binomial}(11,0.50)\) variable is at most 8:
For \(p_T\sim\mathrm{Beta}(2,10)\), the beta cumulative distribution is \(1-(1-c)^{10}(1+10c)\). Therefore:
Under the stated posterior-independence assumption, the joint favorable probability is \(0.9673(0.6779)\approx0.6557\). The safety-trigger probability is \(1-0.6779=0.3221\), below 0.95, and the efficacy evidence is above the futility threshold. Nevertheless, the joint probability is below 0.90, so this illustrative joint rule does not declare success. The appropriate action under these particular gates is continued assessment, not safety suspension or efficacy futility.
Worked Example: A Safety Trigger Overrides Promising Efficacy
In a second hypothetical arm, 8 of 10 participants have the efficacy event and 8 of 10 have the specified toxicity event. With the same priors and thresholds, the efficacy posterior is \(\mathrm{Beta}(9,3)\), so the efficacy probability above 0.50 remains \(1981/2048\approx0.9673\). The toxicity posterior is also \(\mathrm{Beta}(9,3)\).
To evaluate the safety trigger, calculate the posterior probability that the toxicity rate exceeds 0.20. The beta-binomial identity gives the probability at or below 0.20 as the upper tail of a \(\mathrm{Binomial}(11,0.20)\) distribution:
This exceeds the illustrative safety cutoff of 0.95. The arm therefore meets the safety-trigger criterion for suspension, even though its marginal efficacy evidence is strong. The result supports a safety action under the prespecified rule; it does not prove that the true toxicity rate exceeds 0.20. A suspension can allow review of data quality, event attribution, and accumulated safety information before a separate decision about resuming or permanently closing the arm.
Worked Example: Efficacy Futility Without a Safety Trigger
In a third hypothetical arm, 1 of 10 participants has the efficacy event and none has the specified toxicity event. The posterior distributions are \(p_E\mid D\sim\mathrm{Beta}(2,10)\) and \(p_T\mid D\sim\mathrm{Beta}(1,11)\). The efficacy probability above 0.50 is:
This is below the futility cutoff of 0.10. The toxicity probability above 0.20 is \(P(p_T>0.20\mid D)=(0.80)^{11}\approx0.08590\), also far below the safety cutoff of 0.95. Under the illustrative rules, the arm meets an efficacy-futility criterion but not a safety-suspension criterion. It would be misleading to call this a safety failure: the action is based on low posterior probability of meeting the efficacy target.
Derivation and Interpretation of the Calculation
The beta update follows directly from the assumed model. For one endpoint, the prior density is proportional to \(p^{a-1}(1-p)^{b-1}\), and the observed-data likelihood is proportional to \(p^x(1-p)^{n-x}\). Their product is proportional to \(p^{a+x-1}(1-p)^{b+n-x-1}\), the kernel of a \(\mathrm{Beta}(a+x,b+n-x)\) density. This gives the posterior used in the examples; it does not establish that the Bernoulli model or prior is appropriate for a particular trial.
For integer \(a,b\), the binomial-tail identity used above can also be checked by differentiation. The function \(\sum_{k=a}^{a+b-1}\binom{a+b-1}{k}c^k(1-c)^{a+b-1-k}\) has derivative \(c^{a-1}(1-c)^{b-1}/B(a,b)\) and value zero at \(c=0\). It is therefore the \(\mathrm{Beta}(a,b)\) cumulative distribution at \(c\). This identity makes the displayed tail calculations exact apart from rounding.
A posterior probability is conditional on the specified model, prior, and observed data. A rule such as “suspend if the posterior probability of an unacceptable toxicity rate is at least 0.95” means that the current posterior probability of \(p_T>p_T^*\) has reached the prespecified threshold. It is not a 95% frequentist error guarantee, nor does it mean there is a 95% probability that the rule is clinically optimal. The consequences of acting at that threshold depend on the decision context and must be considered when setting it.
Diagnostics, Alternatives, and Limitations
Before applying a rule, check that the data and model match the planned estimand. Confirm that the efficacy and toxicity definitions, observation windows, denominators, and handling of missing outcomes are correct. If participants have not completed an endpoint’s assessment window, treating them as event-free can bias the rate downward. The simple binomial update assumes that each recorded outcome is a relevant observation from the same arm-level probability; strong heterogeneity across sites, changing eligibility, or calendar-time drift can make that assumption doubtful.
Check posterior sensitivity as well as arithmetic. With sparse data, a prior can materially affect a tail probability, so report the prior and examine reasonable alternatives. A beta-binomial model also does not automatically account for association between a participant’s efficacy and toxicity outcomes. When both outcomes are recorded on the same people, inspect their joint pattern and use a joint model if the decision depends on \(P(A_j\mid D)\). Do not multiply marginal probabilities merely because separate beta distributions were fitted.
Separate marginal gates are straightforward to communicate: a safety gate can trigger suspension, and an efficacy gate can trigger futility or support efficacy success. However, passing two marginal gates does not imply a particular posterior probability for the joint favorable region unless the dependence structure is specified. A joint posterior gate directly evaluates that region, but its result depends on the joint model and can be sensitive to assumptions about dependence. A utility-based decision rule is another alternative: it combines the consequences of benefit, harm, and actions into an expected utility. That approach requires defensible values for those consequences and is not equivalent to a probability threshold.
These are arm-level rules, not global platform conclusions. They do not by themselves establish comparative benefit against a concurrent control, account for cross-arm borrowing, or determine platform-wide success. They also use the outcomes observed at the current assessment; they do not calculate the probability of eventual trial success using future outcomes. Those questions require additional design specifications and models.
Common Mistake: Letting Efficacy Evidence Cancel a Safety Signal
A tempting but incorrect argument is: “The posterior probability of efficacy exceeding its target is high, so the arm should continue despite the toxicity result.” This mixes two different claims. Strong efficacy evidence does not reduce the posterior probability of unacceptable toxicity unless the model explicitly links the parameters and the data support that relationship. If the prespecified safety trigger is met, the safety action applies even when the efficacy evidence is favorable. Conversely, low efficacy evidence does not establish that the arm is unsafe. State which posterior quantity triggered each action, and apply the protocol’s priority rule.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Under what condition may two marginal posterior probabilities be multiplied to obtain \(P(A_j\mid D)\)?
- An arm has \(P(p_E>p_E^*\mid D)=0.96\), but \(P(p_T>p_T^*\mid D)=0.97\). If the safety cutoff is 0.95, which evidence triggers an action under a rule that prioritizes safety?
- Why should a participant whose toxicity assessment window is incomplete not automatically be counted as having no toxicity event?
- Explain why an efficacy-futility decision is not the same as a safety suspension.
Solutions. (1) The product is valid when efficacy and toxicity parameters are independent under the joint posterior, so the joint posterior factors into its marginals. (2) The toxicity posterior exceeds the safety cutoff, so the safety-trigger rule calls for suspension even though efficacy evidence is strong. (3) The event could occur during the unobserved remainder of the window; classifying the participant as event-free can understate the toxicity rate. (4) Futility is triggered by low posterior evidence that efficacy exceeds its target, whereas safety suspension is triggered by high posterior evidence that toxicity exceeds its unacceptable-rate threshold. They concern different parameters and different actions.